Headroom: A Local-First Context Compression Layer for AI Agents

Key points

• Problem framing. A single debug session with Claude Code can burn ~65,000 tokens, of which roughly 5% carries task-relevant signal. Beyond cost, long contexts degrade reasoning via the "lost in the middle" effect and incur O(n²) attention overhead.
• Definition. Headroom is a local-first compression layer that processes tool outputs, logs, files, RAG results, and chat history before they reach the LLM, achieving 60-95% token reduction while preserving answer quality.
• Three-stage pipeline.
• *CacheAligner* stabilizes system prompts, tool schemas, and message boundaries so provider KV caches hit reliably. It does not reduce tokens sent, but it makes each cached token meaningfully cheaper.
• *ContentRouter* classifies incoming payloads (JSON, source code, natural language, image, structured logs) and routes each to the appropriate specialist compressor.
• *CCR (Contextually Compressed Retrieval)* keeps originals in a local cache and exposes a headroom_retrieve tool, letting the model request uncompressed content when needed and enabling post-hoc audit of compression decisions.
• Specialized compressors.
• *SmartCrusher* rewrites arrays of JSON objects into a schema-once, rows-many form (50-80% savings on 1,000-row payloads) and handles nested objects, mixed-type arrays, and NULL optimization.
• *CodeCompressor* performs AST-aware, semantics-preserving rewrites for Python, JavaScript, Go, Rust, Java, and C++: comment/docstring removal, safe identifier shortening, unused-import pruning, expression simplification, and structural compaction of implementation details.
• *Kompress-base* is a HuggingFace model trained on agentic traces (tool outputs, chat history, RAG results, logs, error reports) that extracts key information units with importance scoring rather than generating abstractive summaries.
• *Image compression* uses an ML router: OCR-and-decorate-strip for screenshots, downsampling for photos, data-point extraction for charts, and OCR-only for code screenshots, yielding 40-90% size reduction.
• Real-world workload results. Code search 17,765 to 1,408 tokens (92%); SRE debugging 65,694 to 5,118 tokens (92%); GitHub Issue classification 54,174 to 14,761 (73%); code-base exploration 78,502 to 41,254 (47%). Average savings: 60-95%.
• Accuracy retention. GSM8K math accuracy unchanged (0.870 baseline vs. 0.870); TruthfulQA up 0.030; SQuAD v2 and BFCL retain 97% accuracy at 19-32% compression. The framing: compression acts as denoising, not lossy degradation.
• Output-side savings. *Verbosity Steering* appends a compactness instruction at the end of the system prompt (preserving cache prefix matching) to suppress ceremonial phrasing. *Effort Routing* detects routine continuation turns after tool results and lowers reasoning depth, while preserving full depth for novel questions or errors. headroom learn --verbosity personalizes the level. Reported output-token reduction: ~31.7% (95% CI 27.7-35.7%).
• Integration modes.
• *Library*: Python from headroom import compress and TypeScript import { compress }.
• *Proxy*: headroom proxy --port 8787, repoint the agent's API endpoint to localhost, zero code changes.
• *Agent wrap*: headroom wrap claude|codex|aider|copilot auto-configures and launches the agent.
• *MCP server*: headroom mcp install exposes headroom_compress, headroom_retrieve, and headroom_stats to any MCP-compatible client.
• Cross-agent memory. SharedContext with provenance tracking, dedup, TTL, and layered retrieval lets multiple agents (Claude Code, Cursor, Codex) share project facts such as Python version and linter choice.
• headroom learn. Mines past sessions for failure patterns and writes distilled rules into CLAUDE.md or AGENTS.md, providing continuous, project-specific agent improvement.
• Position in the optimization stack. Headroom sits at the *context layer* between model-level efficiency (MoE, AWQ, GPTQ) and agent-level planning (ReAct, Plan-and-Solve). It is the highest-ROI layer because it requires no model swap, no agent rewrite, and no new hardware.
• Local-first rationale. Privacy (sensitive code and logs stay on-device), latency (local inference beats network round-trips on large payloads), cost (no per-call compression API fees), and reliability (works offline once Kompress-base is downloaded; HF_HUB_OFFLINE=1 supported).
• Limitations. Not suitable for fully sandboxed environments without local process execution; redundant when a single provider already offers strong native compaction; risky for tasks requiring 100% information fidelity (legal or medical review) where CCR retrieval mitigates but does not eliminate concern. Compression ratios vary by workload (47% on broad code-base exploration vs. 92% on targeted search). First run requires downloading Kompress-base, and some platforms (notably Intel Mac) may need a local Rust toolchain.
• Conceptual takeaway. Compression is framed as raising information-entropy density rather than discarding content. The piece argues the field is moving from "bigger contexts" to "denser contexts," with Headroom as a second-stage engineering bridge until models internalize selective attention natively.

Reference

• Headroom. *The context compression layer for AI agents.* GitHub: https://github.com/headroomlabs-ai/headroom
• Kompress-v2-base. HuggingFace: https://huggingface.co/headroomlabs-ai/kompress-v2-base