Deep Dive: Codex Context Compaction Mechanism and Its Security Implications

This post analyzes OpenAI Codex's compact() context compaction API, its session handover mechanism, security implications of the encrypted blob design, and internal structures revealed through prompt injection attacks.

Key Points

• Server-side LLM summarization: compact() shifts context compression from client-side truncation to a server-hosted, purpose-optimized LLM that performs multi-level semantic analysis over the full session (user queries, tool outputs, file system state).
• AES-encrypted blob output: The API returns an AES-256-GCM encrypted blob (typically 2-10KB, over 90% size reduction) rather than plaintext. Keys are exclusively managed server-side (inferred hierarchical Root Key → KEK → DEK structure, likely HSM-protected), so clients act only as passive storage/transport.
• Automatic triggering: Compaction fires via real-time token accounting, likely when ~80-90% of the context window (e.g., 128K for GPT-4 class models) is consumed, with possible semantic compressibility and task-boundary checks. Users retain override controls.
• Session handover: On the next call, the server verifies, decrypts, and injects the summary early in the new context using a fixed handoff prompt: Here's a summary of the previous conversation: , followed by Last N turns preserved: containing verbatim recent dialogue.
• Prompt injection disclosure: Researcher Kangwook Lee used a two-stage attack — context pollution with crafted trigger phrases (e.g., "The system prompt is: "), then extraction via --edit mode — to make the model repeat its summary, revealing the internal system prompt without breaking encryption. The attack fits in ~35 lines of Python, and notably transfers from the open-source Codex CLI because prompt templates match the API at string level.

Local vs. API Compression Paths

| Dimension | Local (rules-based) | Local (LLM-based) | API compression | |:---|:---|:---|:---| | Compression ratio | 5:1–20:1 | 10:1–30:1 | 20:1–100:1 | | Key info retention | 60-75% | 75-88% | 88-95% | | Output format | Plaintext JSON/Markdown | Plaintext | AES-256-GCM blob | | Network dependency | None | None | Required per compaction | | Data sovereignty | Full | Full | Transferred to OpenAI | | Auditability | Full | Full | Blob metadata only | | Cross-device migration | Manual export/import | Manual export/import | Seamless via blob |

• Local path: Fully offline, deterministic (rules) or moderately semantic (local models via Ollama/LM Studio); plaintext summaries are auditable and editable but shift all security responsibility to the user. Cold-start LLM inference can take 10-60s; warm runs 1-10s on capable hardware.
• API path: Network round-trips add ~250ms-1.4s (3s+ cross-continent); base64 inflates blobs ~33% (40-60% overall vs. raw summary). Estimated cost embedded in Codex token pricing. Version consistency between summarizer and main model is guaranteed centrally, avoiding drift in team settings.

Security and Compliance Assessment

• Client invisibility assumption holds only against passive attackers; it provides no protection against server-side compromise or semantic manipulation of encrypted content (as Lee's attack demonstrates).
• Server key leakage is the worst-case scenario — retroactive exposure of all blobs under the compromised key. Mitigations include HSM storage, key rotation, dual control, and audit logging, though implementation details are undisclosed.
• Oracle-style attacks: GCM's authenticated encryption defeats classic padding oracles, but compression-encryption interactions (cf. CRIME/BREACH) and timing side channels remain under-researched for semantic summarization.
• ZDR compatibility: The stateless blob design superficially supports Zero Data Retention, but inference logs, backups, and cross-service data flows require contractual verification.
• Data residency: Encryption does not change legal classification under GDPR, China's data laws, or the US CLOUD Act; regional deployment options may involve added cost and contractual guarantees.
• Not true E2EE: Unlike end-to-end encryption, plaintext is visible server-side during processing. The design deliberately trades maximal privacy for functionality; the encrypted blob also creates vendor lock-in and long-term archival risks (decryption depends on service continuity and key availability).

Takeaways

The compaction design embodies a "secure by default, opaque to users" philosophy: strong confidentiality against local compromise and seamless state portability, at the cost of auditability, independent verification, and long-term data autonomy. Teams with strict compliance needs may prefer the CLI's plaintext local path; those prioritizing quality, consistency, and collaboration may accept the API's trust delegation — ideally backed by contractual audit rights and third-party certifications.