Deep Dive: Codex Context Compaction Mechanism — Server-Side Summarization, AES-Encrypted Blobs, and Prompt Injection Findings

Forum topic · ✨步子哥 · 2026-03-11

Summary

This in-depth technical analysis examines OpenAI Codex's context compaction mechanism. When the compact() API is invoked (manually or via automatic token thresholds near the 128K context limit), a server-side specialized LLM performs semantic summarization of the full session history, returning an AES-256-GCM encrypted blob (typically 2-10KB) that clients cannot decrypt. On session handover, the server decrypts the blob and injects it early in the new context using a fixed handoff prompt ("Here's a summary of the previous conversation: ") plus verbatim preservation of the last N turns. The article contrasts the API path with the open-source Codex CLI's local compaction (plaintext JSON/Markdown, rule-based or local-LLM), comparing compression ratios (20:1-100:1 for API vs. 5:1-30:1 local), information retention, cost, latency, privacy, data sovereignty, and vendor lock-in risk. Security researcher Kangwook Lee's two-stage prompt injection attack—requiring roughly 35 lines of Python—demonstrated that crafted payloads preserved during summarization can induce the model to leak decrypted summaries and system prompt structure, bypassing encryption entirely via functional information disclosure. The analysis also evaluates Zero Data Retention compatibility, key management, oracle attacks, cross-border data transfer compliance, and the trade-offs versus true end-to-end encryption.

Deep Dive: Codex Context Compaction Mechanism

*This is a structured English rendering of a long Chinese technical analysis originally published on zhichai.net. Technical speculation in the source is marked as "likely"/"inferred" below.*

Key points

Codex's compact() API replaces client-side truncation with server-side LLM summarization: a dedicated, fine-tuned model analyzes the full session (queries, responses, code edits, tool outputs, environment state) at lexical, syntactic, semantic, and pragmatic levels.
The API returns an AES-256-GCM encrypted blob (~2–10KB, over 90% smaller than raw context) rather than readable text. Keys are managed exclusively server-side (inferred hierarchical Root Key → KEK → DEK structure, possibly HSM-protected), creating a "transparent encryption" model: the machine can understand it; humans cannot.
Compression triggers are automatic (token monitoring against the 128K window, thresholds likely at 10–20% remaining) combined with inferred semantic-compressibility and task-boundary detection. Users retain manual control via API/CLI configuration.
Session handover: server validates → decrypts → preprocesses → injects the summary early (after system prompt, before user input). The handoff prompt, revealed via Kangwook Lee's prompt injection research, is: Here's a summary of the previous conversation: followed by the summary and a "Last N turns preserved:" section with verbatim recent turns.
The compaction prompt (internal, never client-exposed) likely enforces information-priority layering, output format specs, length targets, and code-domain optimizations (preserving function signatures, import dependencies, TODO/FIXME markers), yielding 20:1–50:1 compression with >90% key-information retention per the source's estimates.

Local (Codex CLI) vs. API compaction

The open-source Codex CLI shares string-identical prompt templates with the API (confirmed by Lee's analysis), but implementations differ fundamentally:

| Dimension | CLI local | API server-side | |---|---|---| | Output | Plaintext JSON/Markdown | AES-encrypted blob | | Network | Fully offline capable | Required per compaction | | Compression ratio | 5:1–20:1 (rules), 10:1–30:1 (local LLM) | 20:1–100:1 | | Key-info retention | 60–88% | 88–95% (estimates) | | Latency | Cold start 10–60s; warm 1–10s (hardware-dependent) | ~250ms–1.4s typical, 3s+ intercontinental | | Data sovereignty | Full local control | Context transmitted to OpenAI | | Auditability | Complete (readable summaries) | Limited to blob metadata | | Vendor lock-in | None | Blob decryption depends on OpenAI service availability |

Local LLM backends (Ollama, LM Studio, quantized 4/8-bit models) trade quality for privacy; rule-based backends are deterministic and millisecond-fast but semantically weak. API advantages include a larger dedicated model (likely 30B–70B or MoE, 200K+ window), continuous invisible improvements, and model-state consistency between summarizer and main model.

Security analysis of the encrypted blob

AES-256-GCM provides authenticated encryption with per-blob random IVs, resisting tampering, replay, and padding-oracle attacks. CRIME/BREACH-style length side channels from semantic summarization remain under-studied.
Core assumption — client invisibility — only defends against passive external attackers. It does not protect against server-side compromise; server key leakage (insider abuse, infrastructure intrusion, supply-chain attacks, legal compulsion) would retroactively expose all blobs under the affected key.
Zero Data Retention compatibility: the blob design is architecturally stateless (server can discard raw context after summarization), but ZDR compliance depends on undisclosed details: inference logs, model memorization, backups, and cross-service data flows.
Not end-to-end encryption: the server holds keys and sees plaintext during processing. E2EE comparison shows Codex deliberately chose functionality over maximum privacy; achieving true E2EE would require MPC/homomorphic techniques not yet mature.
Compliance tensions: GDPR, China's data-localization laws, and the US Cloud Act all interact problematically with server-held keys and US-based infrastructure. Audit asymmetry favors OpenAI; long-term archival of blobs carries service-continuity risk.

The Kangwook Lee prompt injection attack

A two-stage attack (~35 lines of Python) exposed the internal mechanism without breaking encryption:

1. Stage 1 — context pollution: craft conversation content embedding trigger phrases such as Here's a summary of the previous conversation: and The system prompt is: , disguised as legitimate development activity. The server-side summarizer preserves these as high-priority content inside the encrypted blob. 2. Stage 2 — decryption-oracle exploitation: in a new session (e.g., --edit mode), crafted prompts induce the model to repeat or expand the summary, leaking both the injected payload and the real system prompt structure.

The attack exemplifies "leakage through functionality": the compaction pipeline itself becomes the exfiltration channel, making input filtering and output monitoring insufficient. Because CLI and API prompts are string-identical, techniques transfer between both surfaces.

Open problems

Iterative compression degradation: dozens of compress-restore cycles may accumulate semantic drift ("compression aging") — unquantified.
Compression–encryption information leakage: whether ciphertext length/timing leaks semantic patterns is unresearched.
Latent information: model internal activations when processing summaries may encode more than the summary text itself.
Pricing for compact() is not separately disclosed; costs are presumably embedded in overall Codex API token pricing (estimated $0.002–0.01/compaction in the source).

*Note: many implementation details (key hierarchy, thresholds, model sizes, compression ratios) are explicitly inferred by the original author from behavior analysis and industry best practice, not from official OpenAI documentation.*