Kiro's Three-Stage Gate: Models Propose, Code Decides

One sentence in Kiro’s tool descriptions made a false claim about how its harness behaves. That sentence reached the model 61,742 times across 445 samples. Nobody filed a ticket. Nobody noticed. An automated tracing system found it, proposed its removal, and a deterministic gate confirmed the removal was safe. The sentence is gone.

That fix is one of seven that Kiro’s internal system, kiro-trace, merged in its first week. Zero were sent back in code review. Each targeted a specific failure pattern measured from real agent runs. The system that produced them is the most rigorous public description of automated agent self-improvement published to date, and its core constraint is worth studying: models propose changes, but code decides whether those changes survive.

The Problem With Self-Grading Loops

Most self-improvement loops let the agent grade its own work. The agent reads its traces, finds a problem, patches itself, re-runs once, sees the number go up, and ships. This fails in predictable ways.

Kiro’s team found one failure mode early. A candidate fix passed more tasks and used fewer tokens. The average looked like a win. When they split results by host, almost all the gain came from a single run where the agent happened to escape a loop it usually gets stuck in. Without that host, the fix was slower, more expensive, and less accurate. A self-grading loop would have shipped it.

The lesson: deciding on averages invites outlier contamination. Deciding on deltas, with a fresh control every time, isolates the contribution of the specific change from variance the model would have produced anyway.

Three Stages, Escalating Cost

kiro-trace runs every candidate fix through a three-stage gate. Each stage is more expensive than the last. Most candidates die at stage one, so the expensive checks rarely run.

Stage 1: Mechanism screen. Runs only on the target failures. Ten runs per arm. The fix must score 10/10 on the targeted task while the control scores 4/10 or below. This is cheap and fast, and it kills the majority of weak candidates before they consume real compute.

Stage 2: Regression screen. Every fix that Kiro has ever confirmed lives as a deterministic assertion in a permanent regression suite. The candidate must hold 100% of those assertions. One failure is a hard stop. This prevents the compounding damage where fix N breaks what fix N-3 repaired, a failure mode that naive loops cannot detect because they check each change only against its own target.

Stage 3: Offline confirmation. The full 89-task benchmark at five runs per arm. The candidate must show superiority on target tasks, superiority on held-out tasks, and non-inferiority within a margin on everything else. A contamination check throws out any pass where the agent used the network to look up the task’s answer instead of working it out.

After all three stages, a human still reviews the pull request. The gate does not ship. The gate decides what reaches a human.

Separation of Roles

The architecture enforces a strict split. Kiro is the agent being improved. The loop reads failures, diagnoses root causes, and proposes one change at a time. The gate, written in plain code, accepts or rejects. A human makes the merge call.

Anything that both writes a change and decides whether it worked will tend to accept it. Kiro’s team learned this from their own experience: while judging comparisons by hand, they once read the same run three different ways. That is why the decision now sits in code the model cannot reach.

What the Fixes Look Like

The seven merged fixes from week one tell a story about what automated trace mining finds that humans do not.

Redundant file re-reads dropped about 83%. The fix: stop re-reading a file that has not changed. Over-strict tool contracts caused 205 read-tool schema rejections and 167 sub-agent orchestration rejections across 80 sampled sessions. Fixed by correcting the contracts. A silent file-search bug broke roughly one in eight searches. The spec returned empty results with no error, so the agent searched and found nothing. Fixed. Over-eager delegation spun up sub-agents for simple direct edits. Fixed by removing blanket delegate instructions.

None of these surfaced as user-visible errors. No ticket pointed at them. A person could have found them, but only by reading far more runs than anyone has time for.

Compaction: From 0.82 to 0.92

The compaction case study shows the loop working from diagnosis through release on a single problem.

Compaction is how Kiro summarizes a long conversation when it hits its context limit so a fresh instance can continue. The old design appended a raw, newest-first dump of past user messages to the end of every summary. When the triggering message was a bare “yes, do that,” its context had been summarized away while an older thread survived word for word. The agent tied the confirmation to the wrong request.

kiro-trace tested 11 candidate summary prompts head to head : two from Kiro, five from production coding agents and public handoff practices, four of their own design. The metric was retention: could a fresh agent, given only the summary, continue the work? The score covered whether the summary was actionable, whether file paths survived, whether earlier decisions were stated, and whether the thread held together.

The winning design carried less context, not more. A short summary written to a terse fixed template, capped in size, plus the most recent turns. No raw message dump. No task-naming in the resume cue. Retention rose from 0.82 to 0.92.

The Unshipped Result

The largest change has not shipped yet. By removing aggressively, the loop built a smaller tool-and-prompt profile that scored 82.4% pass@1 against 77.5% on Terminal-Bench 2.1, at roughly a third of the token cost. It has not shipped because changing the default tools for every Kiro user is a product decision, not a benchmark one. The team is deciding which removed tools should come back only when a task needs them.

This is a useful signal about the maturity of the gate. A system confident enough to hold back a 4.9-point improvement on cost grounds is a system that treats shipping as a product decision downstream of measurement, not as the measurement itself.

Why This Matters Beyond Kiro

The pattern generalizes. Every team running agents in production accumulates prompt and tool debt: instructions that no longer help, tool descriptions that lie, contracts that reject valid inputs on technicalities. The volume of runs makes manual review impossible. The fix is not to let models grade their own work. The fix is to build deterministic gates that separate proposal from decision.

Kiro’s three-stage design offers a template. Screen cheaply on mechanism. Screen permanently on regressions. Confirm expensively on the full benchmark. Turn every confirmed fix into a regression assertion so the suite grows with each improvement. And keep the merge decision with a human.

We have written about self-improving agents and the audit gap in self-healing systems. Kiro’s publication is the first public evidence that the deterministic-gate approach works at scale, with numbers attached.

---

This analysis synthesizes How Kiro Fixes Itself (Myeongsoo Kim et al., Kiro/AWS, October 2026).

Victorino Group helps teams build deterministic gates for agent self-improvement. Let’s talk.