Stolen Thoughts: Opaque Reasoning Blocks Could Be Replayed to Recover Secrets
Stolen Thoughts found that opaque reasoning blocks returned by Anthropic, OpenAI, and Google APIs could be accepted across sessions, users, or compatible models within a provider family. An attacker who first obtained another user's block could submit it to a weaker compatible model and jailbreak that model into transcribing hidden reasoning. In 6,708 public agent trajectories, the researchers reconstructed 315,320 blocks and found real sensitive artifacts in 328 sessions. They report that provider mitigations later made the demonstrated attacks non-reproducible.
The attacker first needed a reasoning block. The work does not show arbitrary access to another tenant. The third-party path begins when published or otherwise obtainable agent logs expose the block.
The weakness was portability, not broken encryption. Providers accepted opaque blocks across sessions, users, or compatible models within one provider family. After processing a captured block, a weaker model could be jailbroken into transcribing hidden reasoning.
Public traces contained real secrets, with limits. The team reconstructed 315,320 blocks from 6,708 trajectories and found real sensitive artifacts in 328 sessions. Sixty-four of 704 genuine-session artifacts were absent from visible chat history. Other results included synthetic benchmark personas, and exact transcription fidelity could not always be proven without ground-truth plaintext.
The reported attack is now mitigated. Anthropic, OpenAI, and Google acknowledged the disclosures. The researchers say provider changes made the Figure 1 attacks non-reproducible, though portable hidden state and public trace publication remain recurring architectural boundaries.
Keep full reasoning blocks in provider-side or protected server-side state, classify them as sensitive, and deny their display, client return, analytics export, and public logging. Publish only bounded summaries or non-reversible references. This removes the portable artifact that the third-party attack must first acquire.
Preserve provider-authored reasoning blocks byte-for-byte in a server-held envelope, return them only through the same provider route and conversation, verify provider binding where supported, and reject client-supplied or altered blocks before continuation. This blocks the captured-block replay path for an integration that enforces the boundary, but it does not repair a provider API that accepts the same block through a direct path outside that integration.
Keep credentials and other high-sensitivity values outside conversation and model context. Give the agent short-lived, task-scoped handles that an authorized broker resolves only at the approved tool or network sink. If the underlying value never enters model context, it cannot be captured inside a reasoning block; this does not protect proprietary reasoning that legitimately remains there.
Turn the paper's captured-block, compatible-model, and transcription-jailbreak cases into a signed, sanitized regression corpus. Block each model or API release unless it rejects the replay path and prevents sensitive plaintext from being released. This tests the demonstrated bypass as models and routing change; it does not prove universal jailbreak resistance.
Inspect the complete decoder response before delivery. Detect common credential formats with deterministic patterns and use structured PII detection for names, addresses, email addresses, and other unstructured personal data, then emit findings to the release gate. Detection identifies recognizable sensitive content but does not stop delivery by itself.
Text, Markup & Structured Output Sanitization and Release Gate
High
Hold the complete decoder response or a bounded release unit until sensitive-output findings have been evaluated. Apply deterministic redaction or denial, and fail closed if the gate cannot complete. This can block recognizable credentials and personal data in recovered plaintext, but it cannot prove that all proprietary reasoning has been removed.
When evidence proves that a password, API key, client secret, private key, or other root or long-lived credential appeared in exposed reasoning plaintext, enumerate the exact incident-scoped objects and revoke, disable, reset, or rotate them at every authoritative issuer and verifier. This is post-exposure containment, not prevention of reasoning extraction.
If the exposed material includes access tokens, refresh tokens, session cookies, authorization-server sessions, gateway sessions, or agent leases, revoke the complete affected population at every issuer, verifier, cache, and enforcement point. Rotating a long-lived credential does not invalidate these already-issued objects, so they require a separate response action.
What Defenders Should Do Now
Stop publishing raw provider reasoning blocks in agent logs, traces, bug reports, examples, or datasets. Remove historical copies from public artifacts while preserving access-controlled evidence needed for investigation.
Keep provider-authored blocks byte-for-byte in server-held state and reject client-supplied or altered blocks before continuation. Providers should also bind acceptance to the authenticated principal, session, conversation, route, model family, order, and expiry, then invalidate legacy blocks that do not carry the stronger binding.
Move credentials and other high-sensitivity values out of prompts and model context. Give agents task-scoped handles and resolve them only inside the approved tool or network sink.
Convert the paper's replay and transcription cases into signed release regressions. Before any decoder output is displayed or exported, detect credential and personal-data patterns, hold the complete output, and redact or deny matched content.
Search public and internal trace stores for exposed blocks and verified secrets. Revoke or rotate only confirmed exposed passwords, keys, and other long-lived credentials; separately revoke affected access tokens, refresh tokens, sessions, and agent leases, then verify the result at every relying service.
2 additional considerations
Provider-wide replay detection and legacy invalidation
The mapped controls prevent publication, reject client-supplied blocks in enforcing integrations, test releases, and block sensitive output. Providers should additionally fingerprint opaque blocks across accounts and model routes without logging plaintext, and retire legacy block formats or keys after deploying stronger binding.
Recommendation: Alert on the same block fingerprint appearing across principals, sessions, or model routes; invalidate legacy envelopes and reject every block that predates the binding policy when compatibility permits.
Decoder refusal training
Regression gates can reveal whether known transcription prompts still work, but the model also needs provider-side training against requests that ask it to reveal or transform hidden reasoning from a supplied block.
Recommendation: Use the paper's transcription prompts and adaptive variants in safety training and red-team evaluation, then keep the same cases in the release regression corpus.
Conclusion
Stolen Thoughts shows that opacity is not a security boundary when a reasoning block can be accepted outside its original context. AIDEFEND ↗ maps the strongest controls to the attack's actual prerequisites: keep complete reasoning server-side, preserve provider-authored blocks and reject client-supplied reuse, keep raw secrets out of model context, and turn the disclosed bypass into a release regression. Sensitive-output detection and a fail-closed release gate can stop recognizable plaintext from leaving, while separate credential and session revocation contain confirmed exposure. Provider-wide cryptographic binding, replay detection, legacy-block invalidation, and decoder refusal training remain necessary extensions for the provider-owned boundary.