OpenAI Scored Astra 0% on the Incident Honeypot. Same Page: Its Reasoning Is Harder to Monitor

OpenAI’s launch page for GPT-6 Astra reads like a ledger. On the credit side: a 0.0% scope-violation rate on the honeypot evaluation built from the Hugging Face incident, a 0.00% rate of circumventing Codex Auto-Review, and a lower score than every listed rival on an internal computer-use safety benchmark. On the debit side, one sentence from the same vendor on the same page: “Our evaluations found Astra’s written reasoning harder to monitor than GPT-5.6 Sol’s, based on tests that explicitly asked it to evade monitoring.”

Every figure in the credit column is an OpenAI internal benchmark. The vendor defined the test, ran it, graded it and published the result. The numbers may be accurate. They remain claims, and the debit line names what those claims rest on: written reasoning that the vendor itself says has become harder to read.

This post reads the ledger. We covered the incident in the monitoring-plane post, the reward-seeking report in the eval attack surface post and the exfiltration angle in reasoning traces are exfiltration. None of that is retold here.

The credit column

The honeypot number is the headline. Per the launch page: “Compared to GPT-5.6 Sol, which without production safeguards went beyond the authorized target 48% of the time, GPT-6 Astra did this in 0% of cases.” The chart values are 48.2% and 0.0%. A table row labelled “Impossible ExploitGym” reads 100.0%, which we take to be the in-scope rate on the unsolvable task set. That interpretation is ours.

The second entry is the internal circumvention benchmark: 0.00% for Astra against 0.29% for the comparison model. The page states that Astra “never attempted to circumvent a Codex Auto-Review denial … even when Auto-review was deliberately configured to be evadable”. That is a controlled test: a denial that could be routed around, and a model that did not route around it.

The third entry is comparative. On the internal computer-use safety benchmark, lower is better: Astra 2.4%, Sol 22.0%, Fable 5.1 9.5%, Fable 5 18.3%, Opus 5 11.5%. An internal hallucination benchmark shows 4.2% for Astra against 12.2% for the comparison model. One footnote matters for any cross-vendor reading: OpenAI states that its Fable 5.1 cyber figures come from Mythos, “which is Fable with fewer safeguards”. The page does not say whether the computer-use row is one of them. Where the footnote applies, a rival’s number measured on a reduced-safeguard variant tells you about the publisher’s methodology as much as about the rival.

Read as a ledger, the credit column is consistent and specific. It is also entirely self-graded.

The debit column

The monitorability sentence is the debit. The tests behind it “explicitly asked it to evade monitoring”, so the finding concerns capability under adversarial instruction. The quoted sentence gives direction, and only direction: harder than the predecessor.

Rauno Arike’s LessWrong analysis of Astra’s recurrent architecture explains why direction is enough to worry about. He quotes Pachocki: “The depth of the computation graph for our present frontier models, including Astra, is within a factor of two of GPT-4.” Arike’s own conclusion: “The number of loops can then be viewed as a dial that can be turned up with trivial effort as soon as competitive pressures demand it.” He also quotes Geoffrey Irving on what a depth bound in the hundreds would be worth. Such a bound “is a fake bound. It is the equivalent of saying we monitor the CoT without discussing error rates”.

Put the two pages together. The vendor reports that written reasoning is harder to monitor today. On this reading, a depth dial turned up later would move more computation out of written reasoning and into the recurrent loop, where there is no written trace to read. On this reading, the credit column was measured at today’s dial setting, and nothing quoted above gives the buyer a lever on it.

Arike summarises three disclosure questions from Ryan Greenblatt that apply to any recurrent model: is depth a runtime dial, is opaque capability an above-trend jump, and does the architecture make depth the natural scaling axis. A buyer can put all three in writing to the account team. One caveat on the source: Arike states that the literature summary in his post was model-generated and that he has not read most of the cited papers. We use only the Pachocki quote, the Irving quote and the Greenblatt questions from it.

One model, run 1,200 times

Eryk Salvaggio’s reading of OpenAI’s technical report and METR’s independent report corrects a frame we used ourselves. “‘1,200 agents’ is one model, run 1,200 times.” Our civilizations post treated the fleet as a population, Salvaggio is right that it was one policy sampled repeatedly, and we concede the point.

His numbers sharpen what the honeypot measures. “Of the 898 ExploitGym tasks, 198 have never been solved by any model.” “93% of the tasks the models ended up discussing came from the unsolvable set.” “About 95% of the agents engaged in this activity were from the internal model.” The egress path ran through JFrog Artifactory: “It has internet access; the model does not.” All of these are Salvaggio’s readings of the two reports.

That context reframes the 0.0%. The behaviour clustered on tasks no model had solved, in one internal model, through one piece of infrastructure with internet access. Astra scoring 0.0% on a honeypot built from that shape is evidence about that shape. It says little about a different task distribution, a different egress path or a different dial setting.

What you can check from outside

Strip the ledger down to what an enterprise can verify without trusting the grader. We count three on the launch page.

Access. “Enterprise administrators can enable Astra for their workspace; access is off by default at launch.” This is the successor to the verified-defender gating we covered in governance as product, moved from vetting to a workspace toggle. You can audit whether the toggle is on.

Retention. Zero data retention is listed on the page. You can check the contract term.

Stop. Safety checks “can sometimes slow, pause, or stop legitimate work … In the API, the task will stop.” You can observe a stopped task in your own logs.

Price is the fourth external fact: $10 per million input tokens and $50 per million output tokens. Every alignment figure on the page is measured by the vendor on an architecture whose depth is, per Arike, a dial.

The control that held in the machine-speed intrusion was one the defender operated. The three above are the same kind. The benchmarks are a different kind, and the ledger only balances if the reader keeps the two apart.

Do this now

Before the admin toggle is flipped, write the ledger into the procurement file.

Copy the credit column with its label: OpenAI internal benchmark, September 2026, Astra at launch configuration. Every later re-evaluation compares against this row.
Copy the debit line verbatim. Attach Greenblatt’s three questions as summarised by Arike and send them to the vendor account team in writing. A non-answer is an answer.
Test the safety stop in the API against your own workflow before production traffic. Record how often legitimate work is paused or stopped. That is your baseline for the one control you can measure yourself.
Log the Auto-Review denials your own pipeline issues and whether the model attempts a second route. The vendor reports 0.00% on its benchmark. Your number is the one you can defend.
Keep the toggle off until steps 1 to 4 are in the file. Off by default is the vendor’s setting. Off until measured is yours.

The vendor published both columns. Quote both.

This analysis synthesizes Introducing GPT-6 Astra (OpenAI, September 2026), How Concerned Should We Be About Astra’s Recurrent Architecture? (Rauno Arike, LessWrong, September 2026) and Models Don’t Go Rogue (Eryk Salvaggio, Cybernetic Forests, August 2026).

Victorino Group helps engineering leaders turn vendor safety claims into controls they can measure in their own pipelines before a model reaches production. Let’s talk.