Headroom: A Local-First Context Compression Layer That Cuts AI Agent Token Usage by 60-95%

> Project: Headroom — The context compression layer for AI agents > GitHub: https://github.com/headroomlabs-ai/headroom > Stars: 51k | License: Apache 2.0 > Key metric: 60-95% token reduction with no significant accuracy drop on benchmarks

The Problem: Expensive, Noisy Context

A typical debugging session with Claude Code can consume ~65,000 tokens (15,000-line logs, 20 code files, 5,000 lines of test output, 100 search results) — roughly $0.195 per session at Claude 3.5 Sonnet input pricing, and thousands of dollars per month for teams. Worse, only ~5% of that context is typically useful. Long, noisy contexts also trigger the "lost in the middle" problem, degrading reasoning quality.

What Headroom Is

Headroom is a local-first context optimization layer that intelligently compresses what AI agents read (tool outputs, logs, files, RAG results, conversation history) before it reaches the LLM — up to 95% token reduction with unchanged answer quality. Its philosophy: different content types have different information density, so each gets a specialized compressor.

Three-Stage Pipeline

1. CacheAligner — Stabilizes prompt prefixes (system prompt formatting, tool schemas, message boundaries) so provider KV caches actually hit. Cached tokens are far cheaper than uncached ones on Anthropic/OpenAI models. 2. ContentRouter — Classifies content (JSON, code, text, image, logs) and routes it to the best-suited compressor, since JSON, code, natural language, and images are redundant in different ways. 3. CCR (Contextually Compressed Retrieval) — Keeps originals in a local cache and gives the LLM a headroom_retrieve tool, so nothing is truly lost. Also enables auditability: you can trace exactly what compressed content the model received.

Specialized Compressors

• SmartCrusher (JSON): Extracts a schema so repeated key names appear once instead of per-object — 50-80% savings on large arrays.
• CodeCompressor (AST-aware): Semantics-preserving rewrites — removes comments/docstrings, shortens locally-scoped variable names, drops unused imports, simplifies expressions. Supports Python, JavaScript, Go, Rust, Java, C++.
• Kompress-base (text): A HuggingFace model (https://huggingface.co/headroomlabs-ai/kompress-v2-base) trained on agentic traces; extracts high-importance information units and discards noise (timestamp precision, deep stack frames, filler language).
• Image compression: ML-routed strategies — OCR for screenshots/code images, resolution/color reduction for photos, data-point extraction for charts. 40-90% size reduction.

Real-World Numbers

| Workload | Before | After | Savings | |---|---|---|---| | Code search (100 results) | 17,765 | 1,408 | 92% | | SRE incident debugging | 65,694 | 5,118 | 92% | | GitHub issue triage | 54,174 | 14,761 | 73% | | Codebase exploration | 78,502 | 41,254 | 47% |

Benchmarks: GSM8K unchanged (0.870 → 0.870), TruthfulQA slightly improved (0.530 → 0.560), SQuAD v2 at 0.97 with 19% compression, BFCL (tool use) at 0.97 with 32% compression. The compression behaves like denoising rather than lossy reduction.

Output-Side Savings

• Verbosity Steering: Appends a concise-output instruction at the *end* of the system prompt (preserving cache prefix matching) to suppress ceremonial filler — important since output tokens cost ~5x input on models like Opus.
• Effort Routing: Detects routine follow-up turns (e.g., after tool results) and lowers thinking depth; full depth is kept for new problems and errors.
• Measured output token reduction: ~31.7% (95% CI: 27.7%–35.7%).

Four Integration Modes

• Library: compress(messages, model=...) in Python or TypeScript.
• Proxy: headroom proxy --port 8787 — zero code changes; point your agent's API endpoint at localhost.
• Agent Wrap: headroom wrap claude|codex|aider|copilot — one-command setup.
• MCP Server: headroom mcp install — exposes headroom_compress, headroom_retrieve, and headroom_stats tools to any MCP client.

Additional Features

• Cross-Agent Memory: A shared context store (SharedContext) with provenance tracking, deduplication, TTLs, and layered retrieval, so Claude Code, Cursor, and Codex can share project knowledge.
• headroom learn: Mines failed sessions and writes lessons into CLAUDE.md / AGENTS.md as reusable rules — a continuous-learning mechanism.

Limitations

• Not usable in fully sandboxed environments that can't run local processes.
• Less valuable if your single provider already offers native compaction.
• Compression-sensitive tasks (legal, medical) carry inherent risk, though CCR mitigates it.
• Compression ratios vary by workload (47%–92%); first run requires downloading the Kompress-base model; some platforms need a Rust toolchain.

Takeaway

Headroom sits in the context layer of the token-optimization stack — the easiest to deploy with the highest ROI (no model changes, no agent rewrites, no new hardware). Its deeper lesson: in the token-economics era, compression is a core AI engineering skill. Refined, high-entropy-density information beats padded, exhaustive context — more context means more cost, more latency, and more "lost in the middle" risk, but not better reasoning.

References

1. Headroom. *The context compression layer for AI agents*. GitHub. https://github.com/headroomlabs-ai/headroom 2. Kompress-v2-base. HuggingFace. https://huggingface.co/headroomlabs-ai/kompress-v2-base