Three Decision Models Shipped. Not One Decides When to Abstain.

Kev-27B accepted 54% of questions at 5.5% observed error. Kev-9B accepted 36% at 6.7%. Those two lines come from the page where Jared Palmer introduces Kev, and they are the only place we found, across three October 2026 launches, where deferral is priced. Cloudflare’s Clef, Kev and the Strands Agents project’s Decider 2B are three decision models, and in the page text we read, none of them decides whether to abstain.

A disclosure first. TypeSafe’s Jev is a tool Victorino uses in its own content-discovery pipeline. Two of the three pages below also claim compatibility with its API, so we have a stake in how the category is judged.

Scope matters here. We read three pages, each twice, with no discrepancy between readers. All three are run by the vendor or the author, and no page reports an independent runner. Every absence below is an absence in the page text, not a claim about what the models can do.

Where Each Page Puts the Abstain Decision

Cloudflare’s Clef post says a decision model “will return typed answers with probabilities (outputs), which your code can use to route the ticket, trigger an escalation, or defer to a human.” The deferral happens in your code. The page describes no abstain output inside the model.

Kev’s page is blunter. “Kev only scores the options you supply. It doesn’t generate explanations or retrieve missing facts.” And: “Your code sets the threshold.” The case where none of the supplied options is correct is left to the caller.

Strands says its decision models “always produce an answer from the selected options.” Deferral lives in the framework: the intervention handler returns Proceed, Deny, Confirm or Guide. The page also says its example “is an illustration rather than a recommendation, so the questions, the threshold and the policy were all picked by hand.”

Three pages, one pattern. The model answers, and the caller decides what to do with an answer it may not be able to trust. That is a reasonable division of labor. In our view it also moves the part of the design that matters most for a gate out of the artifact being benchmarked and into the caller’s code, which none of the three pages ships.

What Kev’s Page Measures That the Others’ Text Does Not

Kev’s author describes the procedure: “I chose a threshold for each model targeting 5% error on development data, then applied it unchanged to test.” The results:

• Kev-27B accepted 54% of questions with 5.5% observed error.
• Kev-9B accepted 36% with 6.7% error.
• Kev-4B accepted 31% with 3.5% error.
• Kev-0.8B accepted 15% with 5.4% error.

Read the table as a trade. Pick a target error, and the model tells you how much work it will take. By simple arithmetic on those lines, three of the four models landed above the 5% target on test data. The page also warns: “A 90% confidence threshold doesn’t imply 90% accuracy. Measure both.” And: “Those error rates won’t necessarily hold on your data.”

The same page reports calibration as pooled error of 0.019 across 14 datasets and 0.073 averaged per dataset for the 27B model, and adds “Calibration varies by workload.” Its author also discloses that “the results come from my own harness, not an independent leaderboard,” and that “earlier candidates’ test results were known before I selected the released 27B, so selection was not fully blind.” This is a self-reported table with its weaknesses written next to it. It is still a table.

Clef’s page describes training “paired with a Brier loss to refine probability calibration.” In the page text we read, it reports no calibration metric: no ECE, Brier or reliability figure. It does say that “a human does not necessarily need to be in the loop for agentic decisions anymore.” In the page text we read, a claim that strong about human review sits beside no calibration number. Strands’ page ranks Decider 2B “3rd of 33 in the 2B class” by linking to a third-party leaderboard, and its accuracy and Brier figures appear only in images our readers could not parse, so we do not use them.

Read the Table Against the Claim

A second check on the Clef page, unrelated to abstention. It claims Clef “beat the decision models on latency (except for Laya…)”. Its own table shows a median of 209.3 ms for Clef, against 84.4 ms for DiffusionGemma Jev and 51.4 ms for Kev-9B. Only Clef-flash, at a 38.8 ms median, supports the sentence. Cloudflare also scored a competitor it chose (Kev-9B), and says “We shortlisted some evaluations below.” Do not compare numbers across these three pages. Each page reports its own numbers from its own setup, and none reports an independent runner.

Compatibility claims deserve the same reading. Clef calls its models “fully Jev-API compatible” and shows a request posting to Cloudflare’s own endpoint. Kev says applications built with TypeSafe’s SDK “can switch by changing the endpoint and model name.” In the page text we read, neither shows a conformance test, and Strands makes no such claim. Our reading of the three pages is that no shared standard is established by them. Two assertions of compatibility with no test between them do not make one.

The Table We Propose

What follows is our proposal, not a claim from any source.

When a decision model sits as a gate, report it as a table of accepted fraction against observed error, at a threshold fixed before testing. Kev’s page is the closest example we found: four rows, one threshold rule, an error target set on development data and applied unchanged. A reader can see what the model costs in deferrals at a given risk.

The form we suggest has five fields per row:

1. The model and size.
2. The target error and the dataset it was fixed on.
3. The accepted fraction on held-out data.
4. The observed error on that accepted set.
5. Who ran the harness, and whether the selection was blind.

A model published without that table should be treated as unmeasured as a gate. It may be an excellent classifier. The point is narrower: the page does not let a reader tell how often it would hand a case to a human, or how often the cases it accepts are wrong.

Related reading from us: a decision model on the control path, the escalation trigger and autonomy scores and revocation.

Do This Now

Before you wire any of these three models into a gate, build the table yourself on your own traffic.

Fix a target error on a development slice. Choose the threshold there and freeze it. Run held-out data through the model once. Record the accepted fraction and the observed error, per model size you consider. Then write down what your calling code does with the rejected share, because the pages leave that to you. If a vendor will not give you a table like Kev’s for its own model, ask for one before you route anything important through it.

---

This analysis synthesizes Introducing Clef: our open-source decision models, and new RL fine-tuning platform (Cloudflare Blog, October 2026), Introducing Kev (Jared Palmer, October 2026) and Introducing Strands Decider 2B: a small, open source, decision model (Strands Agents, October 2026).

Victorino Group helps engineering teams measure the decision models that sit on their agents’ control paths before those models go live. Let’s talk.