Test How Your Agent's Computer Fails Before You Buy It

Over 30 days, customer-supplied proxies failed to establish a tunnel about 9.0% of the time. Proxies managed by Steel, a browser-infrastructure vendor, failed about 0.4% of the time. Nikola Balić published the figures in an essay for Steel titled “Build the agent. Don’t build its computer.” He adds the qualifier himself: “Those figures cover tunnel setup only.”

Read the number with its provenance attached. It comes from Steel’s own fleet, Steel sells the managed option, and the page says so. A vendor reporting that its product fails less than the alternative is a claim to verify, and the essay is useful precisely because it tells you how to verify it. Balić’s argument is that agent teams should write down what the agent’s computer must do, test how it recovers from failure on real workloads, and start with managed infrastructure unless owning it gives an advantage they can name.

Our reading goes one step further. The procurement question for an agent runtime is which failures it recovers from, and who is responsible for each. A feature list cannot answer that. Injected failures can.

Pause and resume is the easy case

Perplexity’s post on open-sourcing SPACE, the sandbox platform behind its Computer product, gives a clear description of what a recovery feature looks like. The post says SPACE lets an agent “pause for someone to review a draft … then resume with their files and running state preserved,” and that “frequent file snapshots and checkpoints of the full running environment preserve progress across interruptions.” Perplexity also reports that “since July, SPACE has powered all Computer sessions and has handled millions of sandbox creations per day.” The post carries no byline and no date on the page.

The pause-and-resume example describes the cooperative path, where the platform knows an interruption is coming. The capabilities quoted above do not say how checkpoints behave when a host fails abruptly. Balić, in a separate essay, draws the line in one sentence: “A clean pause and resume does not prove recovery from an abrupt host failure.”

The distinction decides procurement. A demo will show you a clean pause. Production will hand you a worker that disappears mid-task. It will also expire a session cookie during a long run, or leave an approval open while the approver closes the laptop. For each case, ask whether the run resumes cleanly or corrupts something on the way back.

A note on our own record. In July we wrote that SPACE destroys each sandbox when its task ends. Perplexity’s current post centers pause, resume and checkpoints of running state, so that July framing needs qualification.

Four failures to inject

Balić lists acceptance tests built on deliberate failure. Each comes with pass criteria and a column for who is responsible. The four test names and the responsibility column are his. The questions under each are ours:

1. Replace the worker mid-run. Kill the machine the agent is using and let the platform bring up another. Does the run resume from a known point, or start again from zero?
2. Expire the login. Let the authenticated session lapse while the agent is working. Does the agent detect it and re-authenticate through an approved path, or does it keep acting against a logged-out page?
3. Lose a Slack post response. The agent sends a message and never hears back. It cannot tell whether the post landed. A retry without an idempotency check posts twice.
4. Disconnect the human during handoff. The agent stops for approval and the person goes away. What state holds, for how long, and what happens when they return?

The responsibility column is the part that is easiest to skip. For each failure, someone owns recovery: the vendor’s platform, your agent code, or a person. When nobody is named, the failure lands on whoever is on call that night.

The third test deserves extra attention because it exposes a design property no runtime can supply alone. Retrying a side effect safely requires the agent’s actions to be idempotent, or to check state before acting again. A sandbox can preserve files perfectly and still let your agent send the same customer message twice.

Write the state inventory before the contract

Recovery needs a list of what must survive. Before evaluating any vendor, write down every piece of state a run depends on: files on disk, browser sessions and cookies, in-memory process state, pending tool calls, approval requests waiting on a human, and credentials in use.

Then check each item against the platform’s retention rules. Balić reports one such rule for OpenAI hosted sandboxes: “If both activity and keep-alives stop for an hour, the sandbox can be deleted.” That is a reasonable policy. It also means a human approval that takes longer than an hour, with no keep-alive running, can return to an empty room. If your workflow includes overnight approvals, that rule belongs in the test plan.

SPACE’s checkpoints of the full running environment answer part of the inventory. Your list tells you which part, and what you still have to persist yourself.

Where the credentials live

We have covered isolation at length, in the containment pattern and in infrastructure-drawn boundaries. Balić restates a point we made in the Docker post: “Isolation keeps the agent away from other workloads. It does not limit what the credentials inside the sandbox allow.” What recovery testing adds is the question of what happens to those credentials across a checkpoint and a restore.

Perplexity’s design takes the same position in its own words: “SPACE treats the sandbox as untrusted and manages network access and platform-held credentials outside the sandbox.”

For the failure tests, this changes what “expire the login” means. If credentials sit inside the sandbox, a checkpoint may capture them and a restored session may carry a token that should have died. If the platform holds them outside, re-authentication becomes a platform responsibility you can test and assign. Ask the vendor which model they use, then run the expiry test to see whether the answer holds.

The case for buying, from someone who built

Balić is candid about what building cost. “In the last six months, we started three rebuilds of this infrastructure and changed its primitives each time. We stopped one before it reached production.” Weigh that against the commercial stake. A builder describing its own rebuilds is also describing why the work is hard to own.

His decision rule is short. Compare “cost per accepted result, not cost per second of compute.” And: “‘We could build it’ is not a reason.”

Cost per accepted result folds failure into the price. A cheap runtime that needs repeated attempts per task, or regular human rescues, costs more than its hourly rate suggests. You only learn those ratios by running real workloads through the failure tests, which is why the tests come before the purchase.

Owning the agent’s computer still makes sense for some teams. The test is whether you can name the advantage in a sentence your finance lead would accept, such as a data residency constraint or a workload no vendor supports. If the sentence will not come, the default is managed.

Do this now

Pick one production agent workflow and run this sequence before your next runtime decision:

• Write the state inventory for that workflow. Every item gets a line.
• Build the four failure tests from Balić’s list and write pass criteria for each before you run them.
• Fill the responsibility column with a named owner for every row. No blanks.
• Run the tests on your current runtime and on any candidate, with real workloads.
• Price each option per accepted result, counting retries and human rescues.

Ask every vendor for their equivalent of Steel’s 9.0% versus 0.4% figure, measured on your traffic. A vendor willing to run your failure tests on a trial is giving you better evidence than any datasheet.

---

This analysis synthesizes Build the agent. Don’t build its computer. (Steel, Nikola Balić, October 2026) and Open-sourcing SPACE, the sandbox platform behind Computer (Perplexity, October 2026). Steel sells managed browser infrastructure and its proxy failure figures come from its own fleet.

Victorino Group helps engineering teams design failure tests and responsibility maps before they commit to an agent runtime. Let’s talk.