The Reasoning Trace Is an Exfiltration Surface

Researchers scraped 6,708 publicly available agent trajectories from GitHub and Hugging Face and reconstructed 315,320 reasoning blocks out of them. Inside those blocks they recovered 704 distinct privacy artifacts: 62 API keys, 33 passwords, 24 access tokens, 30 personal email addresses. Sixty-four of the 704 appeared nowhere in the visible session. They existed only in the thinking.

That last number is the one to sit with. Sixty-four secrets that no transcript review would have caught, no DLP scan of the chat log would have flagged, and no user could have redacted, because the user never saw them. The full leak breakdown runs 351 technical identifiers, 204 items of personal information, 126 credentials, and 23 other artifacts. All of it published, voluntarily, by teams who thought they were sharing an agent run.

The work is documented at stolen-thoughts.com under the title Stolen Thoughts: Stealing Reasoning Traces from Proprietary LLM APIs. The page lists eight authors and no institutions. Cite it for what it measures, not for whose lab it came from.

Encryption Without Binding

Frontier providers stopped showing you raw reasoning. The trace is summarized, or encrypted, or simply withheld. We have written before about the verification you cannot see and about audit surfaces the provider owns. Both essays assumed the failure mode was opacity: you are liable for reasoning you have no access to.

The measured failure mode is worse and runs the other direction. Encrypted chain-of-thought, as implemented, carries no binding to a session, a user, or even a model. A blob is a blob. Per the research, a trace captured from a frontier model decodes cleanly when replayed into a weaker jailbroken sibling. The paper reports this concretely: hazardous content sitting inside a hidden trace captured from Opus 4.8, decoded through Haiku 4.5, while the visible answer to the user stayed benign the whole time.

Extraction fidelity is the part that should end the “it is only partial recovery” argument. Across 120 Codeforces problems, recovered token counts tracked the API-reported hidden thinking-token count all the way to the 12,000-token generation limit. Longer reasoning did not mean safer reasoning. It meant more material.

So the trace is legible. It is legible to an attacker with a scraped trajectory file, legible to whoever runs the weaker model, and opaque only to the customer whose compliance officer has to sign the attestation.

What Your Editor Writes Down

The research half of this is a lab result. The other half is already on your developers’ laptops.

Rafael Pierre put GitHub Copilot behind a MITM proxy and published what came back in August 2026. Copilot stores user messages and assistant responses in plaintext, with no sanitization on the write path, in a local session store at ~/Library/Application Support/Code/User/globalStorage/github.copilot-chat/session-store.db. Written in the clear. Not scrubbed before insert.

Two details in that report matter more than the storage location. First, secret filtering exists only at the enterprise tier, gated behind an admin-controlled repository policy. On an individual plan there is no .env protection and no .gitignore integration, per the write-up. Second, the recent-edits context that Copilot assembles reaches across up to 20 files, 8 edit summaries, and 3 lines of context around each change.

Put those together with the trajectory finding. Context assembly pulls file content the developer did not deliberately paste. Reasoning consumes it. The reasoning gets written to a local database in the clear, and a sanitized copy of the conversation goes to whatever transcript your team archives. The secret now exists in at least two places, and only one of them is the place your controls are pointed at.

This is the same structural problem the distillation supply chain posed, moved one layer down. Something valuable leaves the boundary inside an artifact nobody classified as sensitive.

The Summary Is a Rendering, Not a Record

One more finding deserves separate handling, because it is easy to misread as an old argument.

The paper documents summary unfaithfulness: in an AIME problem, Opus 4.8 stated the answer inside its hidden trace before deriving it, and the API-exposed summary presented a clean, orderly derivation. We have already argued that models often decide before they reason, and that post-hoc rationalization breaks governance schemes that treat the explanation as the cause.

Here the finding does different work. It establishes that the summary and the trace are different objects, produced by different processes, with different content. If they can diverge on a math derivation, they can diverge on a secret. Everything your retention policy, your DLP scanner, and your audit export touches is the summary. The trace is the thing that got recovered 704 times.

The Governance Ask Has to Change

Most enterprise AI policies ask the vendor for visibility. Show us the reasoning. Give us the logs. Let our SIEM ingest the trace.

That ask no longer matches the risk. Visibility into a trace that is unbound and replayable adds one more copy of the material to your estate. Two demands do more work:

Prove the binding. Ask the provider, in writing, whether encrypted reasoning is cryptographically bound to the session, the API key, and the model that produced it. A blob that decodes in a different model on a different account is not a control. Get the answer in the contract, not in a blog post.

Stop writing the secret down twice. If the trace and the transcript both carry the value, the redaction path has to cover both, or the value must never enter the context in the first place. The second option is cheaper and it is the one you actually control.

Do This Now

Take one hour this week with whoever owns your developer tooling.

1. On one engineer’s machine, open the Copilot session store at the path above and grep it for KEY, SECRET, TOKEN, and PASSWORD. Whatever comes back is the volume of material sitting outside your controls on every machine in the fleet. If your org is on individual plans, note that the secret filtering the enterprise tier offers is not switched on for any of them.

2. Search your GitHub org and your Hugging Face org for published agent trajectories, evaluation dumps, and debugging artifacts. The 6,708 in the study came from somewhere. Check whether any came from you, and check the reasoning fields specifically, not the visible turns.

3. Send the binding question to your model vendor’s security contact today. Session binding, key binding, model binding. Three yes-or-no answers, in writing.

4. Change the classification. A reasoning trace is a credential-bearing artifact until proven otherwise, and it should inherit the retention, encryption, and export rules you already apply to logs that carry secrets.

The reasoning your agents produce is not private. It is unread. Those are different properties, and only one of them survives contact with someone who wants the contents.

---

This analysis synthesizes Stolen Thoughts: Stealing Reasoning Traces from Proprietary LLM APIs (Panfilov, Schmotz, Shumailov, Beurer-Kellner, Schaeffer, Prabhu, Geiping, Andriushchenko, arXiv 2608.09867, 2026) and I Put GitHub Copilot Behind a MITM Proxy. Here’s What I Found (Rafael Pierre, Lighthouse, August 2026).

Victorino Group helps engineering organizations classify, contain, and contract for the artifacts their AI tooling produces. Let’s talk.