Catastrophic Remembering: Why CLAUDE.md Files Keep Growing

Forum topic · ✨步子哥 · 2026-08-12

Summary

An arXiv paper titled "Why Does CLAUDE.md Keep Growing? Catastrophic Remembering in Agentic Coding" (Kushal Chakrabarti) analyzes 1,867 GitHub repositories, 1,801 multi-version files, 299,440 version transitions, and 247,694 instruction lifecycles. The data shows 64.3% of CLAUDE.md files grow over time while only 26.6% shrink, with a median of 39 instructions and a 90th percentile of 131—well past the threshold where instruction-following degrades. Three hypotheses are tested: instruction staleness, content fragility, and imperfect recall. Only imperfect recall survives, with deletion risk declining with age (-0.032 per commit) and dropping faster as maintainer count rises (interaction β=-0.021, z=-11.7). The author names this phenomenon Catastrophic Remembering, the dual of Catastrophic Forgetting in continual learning. The proposed solution borrows code comments: Prompt Comments that record outcome-grounded rationale, eliminating 99.3% of excess growth and improving instruction-following by 23.1% on WildIFEval.

Background

A typical CLAUDE.md accumulates instructions without ever pruning them. Across 1,867 GitHub repositories and 247,694 instruction lifecycles, 64.3% of files grow over time while only 26.6% shrink. Median size: 39 instructions. 90th percentile: 131 instructions—well past the threshold where instruction-following begins to degrade. Each commit adds a net +4.9 instructions on average. This is not a bug; it is structural.

Three Hypotheses, One Survivor

The paper proposes three explanations:

1. Instruction Staleness: Old instructions become obsolete. Predicted signature: deletion risk rises with age. 2. Content Fragility: Weak instructions die early; survivors are robust. Predicted signature: deletion risk falls with age, independent of maintainer count. 3. Imperfect Recall: Maintainers forget *why* an instruction was added, making deletion risky. Predicted signature: deletion risk falls with age, and falls *faster* as maintainer count grows.

Results:

Log-slope of deletion risk on age: -0.032 per commit (95% CI [-0.047, -0.019]). Staleness hypothesis rejected.
Gamma frailty model absorbing content fragility: slope remains -0.0355. Fragility explains only 30.8% of the effect.
Multi-author × age interaction: β = -0.021, z = -11.7. More maintainers accelerate the decline in deletion risk.

Only Imperfect Recall predicts all three observations.

> You do not delete an instruction because it is useless; you delete it because you cannot remember why it was there.

Catastrophic Remembering

The paper names the phenomenon: Catastrophic Remembering, the precise dual of Catastrophic Forgetting from continual learning. Neural networks overwrite old knowledge when learning new tasks; agentic coding accumulates instructions that cannot be verified and cannot be safely deleted.

A closed-form model (Equation 5) shows that when mean recoverability ρ̄(a) → 0, excess instruction count diverges—even if the task is stable and the addition rate is constant. After mass rewrites, growth accelerates (4.1% per commit → 4.9% per commit) because rationale is wiped while the growth mechanism persists from a lower base. This is the Ratchet.

A further finding: 76.8% of instruction deaths occur in wholesale rewrites, not in careful line-by-line review. Deleting one instruction requires a reason; deleting all of them requires none.

The Solution: Prompt Comments

Software engineering solved this problem decades ago with code comments—records of *why*, not *what* or *how*. The paper ports the idea to prompts as Prompt Comments.

An experiment inverts IFEval: hidden constraints become a known minimal cover D⋆, making excess size measurable.

| Metric | No Comments | With Comments | Gain | |---|---|---|---| | Excess size (15 steps) | +60.4% | -5.8% | 66.2 pp | | Excess size (51 steps) | +211.3% | +1.4% | 99.3% eliminated | | Instruction-following (WildIFEval) | 50.4% | 62.0% | +23.1% relative |

Ablations show that comments must record what happened. Random-noise comments perform no better than none. Comments that record attempts without outcomes are the worst (+70.0%), because they hand successors an unverified premise to extend. What works is outcome-grounded latent reasoning: the hypothesis behind a failure, the result, and how often it recurred.

Practical Takeaways

1. Annotate every instruction with a # comment explaining origin, incident, or alignment target. 2. Periodically wholesale-rewrite, but require every new instruction to carry rationale. 3. Monitor instruction count. Above ~50 instructions, redundancy is likely; above ~100, instruction-following is already degrading.

A Deeper Point

> Every field is borrowing from AI right now. We should borrow back just as readily. The answer to catastrophic remembering was forty years old and one field over, and the next one might be too.

Lehman's laws of software evolution, code-comment practice, and Architecture Decision Records are mature engineering traditions that have not yet been formally ported to the LLM era. The paper's sharpest line:

"If English is the new code, why don't we have comments yet?"

---

Paper: arXiv:2608.11095 — *Why Does CLAUDE.md Keep Growing? Catastrophic Remembering in Agentic Coding* Author: Kushal Chakrabarti Scale: 1,867 repos / 1,801 multi-version files / 299,440 transitions / 247,694 lifecycles Core finding: Deletion risk falls with instruction age (-0.032/commit) and with maintainer count (β=-0.021, z=-11.7); root cause is imperfect recall, not staleness. Solution: Prompt Comments that record latent reasoning eliminate 99.3% of excess growth and raise instruction-following by 23.1%.