Don't Rewrite Your CLI for Agents: The Tool Interface Is a Control Surface

Swapping a command-line tool’s flag-style arguments for a single JSON payload made agent tasks cost 4 to 11 times more per run, and made the weaker models return wrong answers. That is what Microsoft found after benchmarking one of the most repeated pieces of advice in agent engineering: make your developer tools “agent-friendly” by feeding them structured JSON instead of the --flag value conventions humans use.

The advice sounds reasonable. Agents emit and parse JSON natively, so a JSON interface should fit them better than argument strings built for a terminal. Waldek Mastykarz, a principal developer advocate at Microsoft, tested the claim instead of accepting it. The conventional interface won on every axis a team measures in production.

One Task, Two Interfaces

Mastykarz kept the tool and the task fixed and changed only how the agent called it. One version took ordinary command-line flags. The other took a single JSON payload carrying the same parameters. He ran each version five times against a set of models inside the GitHub Copilot Chat harness, including GPT-5.3-Codex, Haiku 4.5, and MAI-Code-1-Flash. Same prompt, same goal, same five runs. The interface was the only variable.

That design is what makes the numbers usable. When correctness and cost diverge across the two versions, the tool did not change and the task did not change. The way the agent addressed the tool did.

Conventional Args Were Correct Every Time

Start with correctness, because it is the result that should end the debate. With flag-style arguments, every model scored 5 out of 5. Perfect, across strong and weak models alike.

Switch the identical task to JSON and the stronger models held, but the weaker ones broke. Haiku 4.5 fell to 2 out of 5. MAI-Code-1-Flash landed at 3 out of 5. Nothing about the underlying work got harder. The JSON interface simply demanded more from the model to call the tool correctly, and the models with less headroom spent it on formatting instead of the task. Interface choice decided whether a smaller, cheaper model could be trusted at all.

JSON Cost 4 to 11 Times More

Cost moved the same direction, for every model, without exception. JSON ran 4x to 11x more expensive per task. GPT-5.3-Codex went from $0.05 to $0.54 on the same job, an 11x jump. Haiku 4.5 paid roughly 8x more.

The mechanism was retries. When a model produced malformed JSON, the call failed and it tried again. Those retries generated 7 to 14 times more output tokens than the clean single-shot runs the flag-based interface produced. Structured input that reads as tidier on a whiteboard became a token bonfire in the harness, because every escaping mistake bought another round trip.

The Result Nobody Predicts

The finding that should stop teams cold is cross-platform. JSON’s cost depended on the shell it ran in. On PowerShell, the JSON interface cost 9x more than on Bash, because quoting and escaping rules differ between shells and the model kept getting them wrong. Conventional arguments barely noticed the difference: $0.05 on one shell, $0.07 on the other.

An interface that looks platform-neutral introduced a 9x cost swing that surfaces only on some developers’ machines. A team testing on Bash alone would ship it, then watch spend and failure rates spike for every colleague on Windows, with no obvious cause in the logs. Single-environment evaluation hides exactly this class of defect.

The Interface Is a Control Surface

Put the three results together. The interface set correctness, it set cost, and it set platform variance. Same tool, same task, same models throughout. The one thing that changed moved every number a production team watches.

That is the definition of a control surface. The agent-tool interface behaves like a rate limit or a retry policy: a knob that directly sets failure rate and spend. Treating it as a matter of taste ignores that effect. Most teams treat it as a design preference, chosen once on intuition and never revisited. The Microsoft data says it earns the same scrutiny as any other production control. You measure it, you understand its blast radius, and you test it before you change it.

A second lesson sits underneath the first. The winning interface was the one that already existed. Teams were being told to rewrite working tools on the theory that agents need bespoke, machine-shaped inputs. The evidence points the other way. We have argued that agents often perform better with the plain artifacts humans already read than with structures built only for them. The human-readable CLI beating the machine-structured JSON, for machines, is that same pattern showing up in the tool layer.

It also raises the stakes on evaluation surfaces generally. Once you accept that the interface governs cost and reliability, the interface becomes something you have to watch, which is the argument we made for treating agent output as a measurable surface rather than a black box.

Measure Before You Restructure

The reusable asset here is the method. Mastykarz did not win an argument. He ran a cheap experiment that any team can copy before spending weeks on a rewrite.

Pick one real scenario. Choose a single task the tool actually performs in your workflow, not a toy example. The whole point is to test the real thing.
Define correctness up front. Decide what a right answer looks like before you run anything, so scoring is not a judgment call after the fact.
Run both interfaces across your models. Include the cheaper, weaker models you hope to route work to. The strong models often mask an interface flaw the small ones expose.
Run across your shells and platforms. Bash and PowerShell at minimum. The 9x variance lives here, and it is invisible in a single environment.
Read cost and token counts, not just pass or fail. A version can be correct and still cost 11x more through silent retries.

The experiment is inexpensive. A rewrite driven by intuition is not.

Do This Now

Find one tool your agents call today and check whether anyone has proposed making it “agent-friendly” with a JSON interface. Before that work gets scheduled, run the five-step test above on the existing interface versus the proposed one. Score correctness, per-task cost, and shell variance. If the conventional interface holds, and Microsoft’s data says it usually will, you have saved a rewrite and kept a cheaper, more reliable tool. If JSON genuinely wins for your case, you now have evidence instead of a hunch. Either way you have started treating the interface as what it is: a control you tune with numbers, rather than a design you redraw on faith.

This analysis synthesizes Don’t Rewrite Your CLI for Agents (Microsoft Developer Blog, July 2026).

Victorino Group helps teams treat the agent-tool interface as a measurable control surface, not a rewrite. Let’s talk.