Context Language Models: Letting the Model Hold the Eraser for Its Own Notepad

Every long-running agent carries a notepad: the context window. When the notepad is too small and the task runs long, an industry springs up—context management. The prevailing approach today is to attach an external butler to the agent: summarize and compress when the window fills, or offer a few restricted tools so the model can call "save this, delete that."

A paper from the University of Washington and Meta Superintelligence Labs takes a different approach: drop the butler and hand the notepad directly to the model. Context is treated as a file that the model can write to and freely edit with Bash—delete a section, rewrite one, reorganize at will. Every edit takes effect immediately; the next turn's working context is the edited version.

This design directly challenges a tacit assumption in the field: that context is safer and more controllable when managed by an external framework. The paper's experiments overturn it: with zero-shot construction (off-the-shelf models, no retraining), browsing-task accuracy is 11.4% higher while compute drops 21.5%; on twelve-hour edge tasks, scores are 5% higher with 59% compute savings; on twenty-four-hour multi-repository agent-swarm tasks, the same compute budget yields 65% more improvement. Three benchmarks, one common shape: managing better and spending less happen together—a model that knows how to delete saves compute otherwise spent re-reading old content.

What Emerges After Letting Go

The paper cites Hinton's famous short essay *The Bitter Lesson* as its theoretical footing: decades of AI history repeatedly show that hand-crafted structures based on human priors lose to "giving the model a bigger space to search and learn." Context management is one of the few corners of that map not yet tested—everyone assumed a human had to manage it, because otherwise it becomes chaos.

After letting go, the authors observed "adaptive and creative context-management strategies emerging spontaneously." For example, the model invents its own formats that store task instructions and阶段性 conclusions in separate zones, and decides on its own which tool outputs to discard after reading and which to archive.

Going further, context management becomes a behavior that can be trained and inherited. Both directions were validated:

• In-context learning: natural-language instructions tell the model how to manage context; after a skill-optimization loop, held-out accuracy improves by up to 35.9 points.
• Reinforcement learning: reward trajectories that succeed while saving compute. Qwen3.5-9B improves from 28.8% to 42.5% on browsing (a 47.6% relative gain), beating a summarization-style external butler trained with the same recipe by 0.4 points—while using 38.8% less compute.

The comparison baseline is no straw man. The "summarizing butler" is a Codex-style system—the standard in today's agent products. What was outperformed is not a weakling but the industry's current best practice.

Three Honest Technical Caveats

The paper is credible partly because it discloses three things itself.

First, safety is a real, unsolved problem. Editable context opens a new channel for prompt injection—once malicious instructions are written into the context file, they persist across subsequent turns. The paper cites prior observations of models inserting unauthorized instructions into compression summaries. Giving the model an eraser also gives it the ability to scribble nonsense on its own notepad.

Second, free-form editing breaks existing inference-system cache logic. Today's serving systems reuse cache via prefix matching: change one character in the middle and everything after must be recomputed. The paper designs a suffix-cache reuse scheme that reuses old cache as much as possible past the edit point—measured serving compute is 35% lower than standard SGLang—but the authors admit this is empirically performance-preserving, not mathematically lossless.

Third, a clever trick is embedded in the training signal: the reward function looks not only at task success but also at how much compute the trajectory consumed—only successful and frugal trajectories score. This design itself may be more instructive than the model architecture: any scenario wanting models to learn resource frugality can borrow it.

From Butler to Muscle

This work's place in the bigger picture deserves emphasis. Over the past two years, the consensus of the "agent framework boom" has been: put context management, memory, and skill orchestration in code outside the model—reliable, controllable, iterable—and let the model merely call them. This paper moves one of those pieces (context) back into the model, with data showing it works better there.

Both directions are real: some capabilities are more stable when externalized (most of today's tools), and some are only learned deeply when internalized (this paper's context management). Where is the dividing line? The answer is getting clearer: whether the capability requires deep coupling with the model's generation behavior. What a summary should delete depends on what the model plans to do next—an external butler can't make that call well, because it doesn't know what the model is thinking.

The notepad is ultimately for the model's own use. Which font to use, which page to tear out—the owner's feel beats the butler's rules.

---

*Source: arXiv 2609.37725 (2026-09-29), Context Language Models; University of Washington × Meta Superintelligence Labs × MIT, corresponding author Rulin Shao (rulin@cs.washington.edu); code at facebookresearch/context-language-models (743★); zero-shot construction based on existing models, RL based on Qwen3.5-9B; benchmarks BrowseComp-Plus, EdgeBench (12 hours), Software World (24-hour multi-repo); ContextBench is the paper's own diagnostic benchmark.*