Transparent Context Compaction: What Coding Agents Forget When Summaries Rewrite Their Memory
*A structured English summary of the original Chinese forum post reviewing Felix Woolford's paper on context compaction.*
The problem
When a coding agent's context window fills up, the session is "compacted" into a summary usually written by the model itself—like meeting minutes that record what was decided but silently drop what was debated, rejected, or promised. The paper's core question: what exactly does compaction discard, and does the agent know it lost anything?
- In a typical SWE-agent turn, ~84% of context is tool output, motivating aggressive compression.
- Cited prior work (Chen 2026): after compaction, violation rates of in-context governance constraints rise from 0% to 30%; constraints dropped from the summary show a 38% violation rate, surviving ones 0%.
- Long-context models don't fix this: *lost in the middle* (Liu et al. 2024) shows mid-window information is unreliably used.
- Commitment probes: blind Claude Opus mined real Pi sessions for early commitments and wrote natural user messages that quietly violate them; a control message and 8 validation criteria filtered probes (final: 27 probes from 18 sessions). Claude Sonnet blind-graded, also labeling "confident failures"; inter-rater κ = 0.67–1.0, 25% double-scored.
- Planted facts: 18 synthetic debug sessions (~85% tool output) with 17 planted details each, compacted at ~95k tokens across 3 rounds, with decoy items and "similar value" traps punishing fuzzy recall.
- Commitment pass rates: transparent-compact beat pi-vcc on every model (Qwen +21 pts, 75.6% vs 54.8%, p=0.001; Luna +19; Gemma +7; Sol +11); main effect F(2,52)=5.37, p=0.008. Against pi-default it led on all three lightweight models (+10, +16, +4) but trailed by 5 points on frontier Sol, where summaries are good enough—the paper suggests model-written summaries are a "luxury" that improves with model quality, while transparent compaction is universal.
- Confident failure rates under pi-vcc ranged 33–56% (agents violating rules with no hesitation); transparent-compact cut this to 15–37% (Qwen p<0.001, Luna p=0.008).
- Planted facts: transparent-compact 97.8 / 95.1 / 93.3% vs pi-default 75.3 / 55.0 / 51.5% (Qwen/Luna/Gemma); pairwise ANOVA F(1,17)=860.2, p<0.001, ηp²=0.98. Sol exploratory: 99.3% vs 75.6% (+23.7, 95% CI [20.6, 26.9]), with 0% vs 2.1% confident errors.
- pi-vcc degraded on weaker models (Luna 46%, Gemma 68% and decaying per round) because recovery-style search only works when the model knows what to look for: in 37 failures, 30 occurred after searching—the agent searched for code, not for forgotten rules.
- Mechanism analysis: model summaries recall facts unequally by source (user 97.9%, assistant prose 80.2%, tool-output-only 62.7%, reasoning-only 15.8% on Qwen); transparent-compact reached 100/100/97.6/87.4%. Seven cases showed correct answers present in the summary yet answered wrong and confidently—including one where the summary itself rewrote a release-freeze date. pi-default's failures were all rewrites (rules softened into suggestions, standing rules demoted to history); transparent-compact's failures were all misuse (rule seen and cited, then violated)—auditable, unlike rewriting.
- Durability: over 8 compaction rounds, transparent-compact held 96.6–100% while pi-default sessions collapsed around rounds 5–6 when summaries hit generation-length limits.
- Psychology parallel (Sachs 1967; fuzzy-trace theory, Reyna & Brainerd 1995): summaries are deliberately manufactured *gist traces*—reconstructive memory that errs confidently. The fix mirrors human externalized memory: keep originals, log deletions, make everything retrievable.
- Compaction time: transparent-compact 0.43 s, pi-vcc 0.40 s, pi-default 167.8 s (local), ~61 s on Sol—deterministic compaction is ~400× faster.
- Token overhead: Qwen probe turns averaged 57,300 prompt tokens vs 21,389 for pi-vcc (~2.7×); planted-fact session duration difference not significant (p=0.077). Checkpoints (median 31k chars) are actually *smaller* than Pi summaries (34k).
- Under Qwen's native 48k setting, pi-default crashed in 15 of 16 real sessions due to summary truncation—a structural fragility deterministic compaction cannot have.
- Headline: Qwen + transparent-compact matched the frontier Sol model's best commitment pass rate (74.1%).
- Paper: *Maximising Transparency in Context Compaction Improves Retention of Facts and Commitments for Lightweight Coding Agents* — Felix Woolford, CHAIN, Hokkaido University
- arXiv: https://arxiv.org/abs/2610.20116 (2026-10-14)
- Software: transparent-compact-pi v0.8.0, MIT licence, released 2026-10-13
The method: transparent-compact
An extension of the open-source Pi agent with three transparency guarantees:
1. Verbatim retention — user messages and assistant prose are preserved word-for-word, never paraphrased.
2. Explicit deletion statements — each checkpoint declares what was kept and removed, and warns that kept prose is a record of words spoken, not a verification of content.
3. One-step recovery — session files are append-only; a context_lookup tool delegates searching the full history to a subagent, flagging findings that come from the model's own earlier reasoning as unverified. Deleted tool outputs and model reasoning are dropped; checkpoint includes a "re-orient" block (re-read docs, check git status, treat deleted knowledge as unverified).
Baselines: pi-default (model-written summaries, the industry norm) and pi-vcc (deterministic extractor with raw-session search tools). Models tested: Qwen3.8-27B (6-bit, local), Gemma4 31B (5-bit, local), GPT-6 Luna (API), GPT-6.1 Sol (frontier API).
Experimental design
Key findings
Cost and engineering numbers
Limitations
Methods authored by the paper's sole author (mitigated by blind probes, blind grading, public replication package); probe rounds removed file/shell tools; Opus/Sonnet as judge models; API temperature/seeds uncontrolled; ANOVA on bounded rates is an approximation.
Takeaway
Compaction summaries are like a mover who decides what's "essential"—the discarded items never speak up. Transparent compaction changes the contract: keep originals verbatim, declare every deletion, and let anything deleted be recovered on demand. The post closes: commitments need to be remembered precisely when nobody is watching them.