MIT has introduced AM-OMP (Attention Matching - Orthogonal Matching Pursuit), a fast KV cache compaction technique described in the paper *Fast KV Compaction via Attention Matching*.
Core Technical Architecture
1. Theoretical foundation: Attention mixing identity — the final output of concatenated attention blocks is a weighted mixture of each local attention output. 2. Scalar bias compensation — introduces a per-token scalar bias β, solved in closed form via NNLS. 3. Three-step closed-form solution (no gradient descent required):
- Key selection: greedy selection via OMP
- Bias fitting: NNLS to match attention mass
- Value reconstruction: OLS to match attention outputs 4. Non-uniform head budget allocation — different attention heads receive different compression ratios.
- QuALITY accuracy at 50x compression: 0.67 (original: 0.72)
- Compression time: ~30 seconds per document (compared to ~5 GPU hours for Cartridges)
- LongHealth at 60k tokens with 10x compression: ~0.70 accuracy
- Combined with summarization, total compression ratio can reach 200x
- Paper: arXiv:2602.16284 — https://arxiv.org/abs/2602.16284
- Code repository: https://github.com/adamzweiger/compaction
Performance (Qwen3-4B)
Online Compaction Experiment (AIME 2025)
With a physical length of 2048 and an effective length of 8192, the model scored 13/30, on par with standard 8192 decoding.