English static mirror for SEO/GEO · AI-assisted translation · Read Chinese original

Chained Recursive Language Models: Handing Off Context Like Coworkers

Forum topic · ✨步子哥 · 2026-08-06

Summary

Chained Recursive Language Models (Chained RLM), proposed by Purbesh Mitra and Sennur Ulukus, tackle context rot—degradation of reasoning quality when large language models process long contexts in a single pass. Instead of asking one model to read, track, compute, and answer all at once, Chained RLM invokes the same model repeatedly in fresh root calls. Between calls, only compact handoff materials are passed: a summary of progress, a blackboard of key findings and open questions, and task-specific artifacts. Raw prior trajectories are stored separately and consulted only when needed. On GPT-5-mini benchmarks, Chained RLM improved accuracy over a standard LLM by +5pp on RULER, +15pp on BABILong, +11pp on LongBench v2, and +24pp on OOLONG-real, with the largest gains on tasks requiring long-range information aggregation. The trade-off is roughly 1.8x token consumption and doubled per-task cost, but intermediate reasoning becomes auditable, rollback-capable, and correctable. The work reframes the long-context problem from expanding context windows toward divide-and-conquer reasoning. Paper: arXiv:2608.05124.

Have you ever asked an AI to read a 50-page contract and answer a question about a specific clause? It reads everything, says "page 32," and you find its answer is wrong there. Ask again, it gives a different answer. Ask a third time, it changes again.

This isn't because the model is dumb. It's context rot—when the context is too long, the model "forgets" details it read earlier or mixes up information from different parts. Hong et al. named this phenomenon in 2025.

Purbesh Mitra and Sennur Ulukus propose a clean solution in their paper *Chained Recursive Language Models for Multi-Iteration Reasoning* (August 2026): don't make the model do everything in one shot. Let it work in shifts, like coworkers handing off—each shift starts fresh and passes only the necessary work notes.

The Problem: One Inference Does Too Much

Traditional long-context reasoning works like this: stuff everything into the context, and have the model complete in a single inference—read the context, find evidence, track hypotheses, do computation, use tools, and decide when to answer. It's like asking a person to stand before a giant whiteboard, scan it left to right, remember everything important, and give an answer.

The problem: this "simultaneously" is too heavy. Models do fine on short contexts, but once context grows, they start to:

  • Skim shallowly: surface-level extraction without digging deep
  • Forget what was already checked: losing track of what's been verified
  • Finalize too early: answering before finishing a real audit
  • Chain-of-thought, self-consistency, and Tree-of-Thoughts all try to mitigate this, but they operate within a single inference. Chained RLM changes the layer.

    The Solution: Handoffs, Not Overtime

    The core design of Chained Recursive Language Models: the same model is invoked repeatedly, each time as a fresh "root call" that does not inherit the full conversation history of the previous round.

    But inheriting nothing won't work—each round would start from zero. The researchers designed three handoff artifacts:

    1. Summary: A compact plain-text description of "what we've accomplished so far." The briefing for the next shift.

    2. Blackboard: A plain-text workspace recording current key findings, open questions, and intermediate results. The next shift can read, modify, and extend it.

    3. Artifacts: Task-specific persistent outputs. For a codebase task, an artifact might be a "list of identified functions"; for a long document, an "extracted event timeline." These are structured work products—more precise than plain-text summaries.

    The raw predecessor trajectory is stored separately and not passed by default to the next round. Only when the next round finds it needs to look back does it retrieve it. This keeps the context from drowning in verbose history.

    A Concrete Example

    Suppose the task is: given a 200-page earnings call transcript, find every statement management made about AI investment, and judge which are commitments and which are outlook.

    Traditional approach: stuff all 200 pages into context and answer in one shot. The result likely misses half the statements or mixes up different quarters.

    Chained RLM approach:

  • Root call 1: Read pages 1–50, write AI-related statements found to the blackboard, generate a summary: "Processed pages 1–50, found 3 AI statements: 2 outlook, 1 commitment."
  • Root call 2: Receives summary + blackboard + artifacts, reads pages 51–100. It knows what was already found and continues. Finds 2 new statements, updates the blackboard.
  • Root call 3: Reads pages 101–150. Finds a statement contradicting one from root call 1, and flags it on the blackboard.
  • Root call 4: Reads pages 151–200, consolidates all findings, outputs the final answer.
  • The key difference: every root call is "fresh"—the model doesn't fatigue from reading 150 pages, and its attention isn't diluted by having already processed too much. Each time it restarts with concise work notes.

    Experimental Results

    The researchers compared Chained RLM against a standard LLM on four long-context benchmarks (both using GPT-5-mini):

    | Benchmark | Standard LLM | Chained RLM | Gain | |------|---------|-------------|------| | RULER | 87% | 92% | +5pp | | BABILong | 44% | 59% | +15pp | | LongBench v2 | 41% | 52% | +11pp | | OOLONG-real | 14% | 38% | +24pp |

    A clear pattern: the more a task requires aggregating information across long distances, the bigger Chained RLM's advantage.

    The gain on RULER is small (+5pp) because many RULER tasks can be solved with local retrieval. The gain on OOLONG-real is largest (+24pp) because that task requires the model to "retain partial evidence, compare distant events, and aggregate many small observations"—exactly where single-pass inference fails most.

    The Cost: More Expensive, But More Auditable

    Chained RLM is not free. Its token consumption is about 1.8x that of a standard LLM, with costs scaling accordingly. The researchers ran the numbers (based on GPT-5-mini pricing):

  • RULER: standard LLM $0.11/task, Chained RLM $0.21/task
  • BABILong: standard LLM $0.14/task, Chained RLM $0.28/task
The researchers are direct: the extra cost buys more auditable, more correctable intermediate reasoning. Every root call is a unit of work that can be inspected, rolled back, and corrected. That's far more controllable than compressing all reasoning into a single generation.

Why This Matters

First, it shifts the "long context" problem from "stuff more" to "split smarter."

For the past two years, long-context research has been about "expanding windows from 32K to 128K, 1M, 10M." But Chained RLM points out: even with an infinite window, single-pass inference still rots. The problem isn't "can it fit," but "can it be processed well once it fits." This is a layer switch—from scaling up to divide and conquer.

Second, it echoes the principle of "granularity isomorphism."

The granularity of agent system optimization should match the granularity of the object being optimized. The natural granularity of long-document processing is "paragraph-level evidence integration," not "whole document in one pass." Chained RLM refines the reasoning unit from "one complete generation" to "one root call processing one segment," aligning exactly with the task's natural granularity.

Third, it makes intermediate reasoning auditable.

Traditional one-shot inference is a black box—the model reads 200 pages and gives an answer, and you don't know how it reasoned in between. Every Chained RLM root call leaves behind a summary, blackboard, and artifacts. You can inspect how the model understood the first 100 pages at root call 2, and roll back to root call 2's results and re-run root call 3. This is another instance of "division of labor beats unification" applied to inference architecture—not making one model do everything at once, but letting the same model work in shifts and do one thing right each time.

Honest Limitations

The researchers acknowledge: Chained RLM's gains on RULER are limited, showing that for tasks solvable by direct retrieval, the overhead of divide-and-conquer isn't worth it. And the doubled cost is real—not every scenario justifies paying twice as much for auditability and accuracy. The authors also didn't test on all long-context benchmarks; OOLONG-real's 14% → 38% is a big relative gain, but the absolute number is still not high.

Yet this work's value isn't "how much better than single-pass inference"—it's that it points to a path different from "keep expanding the window": make reasoning itself an engineering process that can be divided, handed off, and audited.

---

Paper: Chained Recursive Language Models for Multi-Iteration Reasoning, Mitra & Ulukus, 2026

Code: alexzhang13/rlm (GitHub)

Tags

#large-language-models#long-context#context-rot#recursive-models#reasoning#agent-architectures#auditability#divide-and-conquer

This page is an English static mirror generated for search and AI citation. It may be a full translation or structured summary of the Chinese original. Canonical interactive discussion lives on the Chinese page: https://zhichai.net/topic/178603049