Have you ever asked an AI to read a 50-page contract and answer a question about a specific clause? It reads everything, says "page 32," and you find its answer is wrong there. Ask again, it gives a different answer. Ask a third time, it changes again.
This isn't because the model is dumb. It's context rot—when the context is too long, the model "forgets" details it read earlier or mixes up information from different parts. Hong et al. named this phenomenon in 2025.
Purbesh Mitra and Sennur Ulukus propose a clean solution in their paper *Chained Recursive Language Models for Multi-Iteration Reasoning* (August 2026): don't make the model do everything in one shot. Let it work in shifts, like coworkers handing off—each shift starts fresh and passes only the necessary work notes.
The Problem: One Inference Does Too Much
Traditional long-context reasoning works like this: stuff everything into the context, and have the model complete in a single inference—read the context, find evidence, track hypotheses, do computation, use tools, and decide when to answer. It's like asking a person to stand before a giant whiteboard, scan it left to right, remember everything important, and give an answer.
The problem: this "simultaneously" is too heavy. Models do fine on short contexts, but once context grows, they start to:
- Skim shallowly: surface-level extraction without digging deep
- Forget what was already checked: losing track of what's been verified
- Finalize too early: answering before finishing a real audit
- Root call 1: Read pages 1–50, write AI-related statements found to the blackboard, generate a summary: "Processed pages 1–50, found 3 AI statements: 2 outlook, 1 commitment."
- Root call 2: Receives summary + blackboard + artifacts, reads pages 51–100. It knows what was already found and continues. Finds 2 new statements, updates the blackboard.
- Root call 3: Reads pages 101–150. Finds a statement contradicting one from root call 1, and flags it on the blackboard.
- Root call 4: Reads pages 151–200, consolidates all findings, outputs the final answer.
- RULER: standard LLM $0.11/task, Chained RLM $0.21/task
- BABILong: standard LLM $0.14/task, Chained RLM $0.28/task
Chain-of-thought, self-consistency, and Tree-of-Thoughts all try to mitigate this, but they operate within a single inference. Chained RLM changes the layer.
The Solution: Handoffs, Not Overtime
The core design of Chained Recursive Language Models: the same model is invoked repeatedly, each time as a fresh "root call" that does not inherit the full conversation history of the previous round.
But inheriting nothing won't work—each round would start from zero. The researchers designed three handoff artifacts:
1. Summary: A compact plain-text description of "what we've accomplished so far." The briefing for the next shift.
2. Blackboard: A plain-text workspace recording current key findings, open questions, and intermediate results. The next shift can read, modify, and extend it.
3. Artifacts: Task-specific persistent outputs. For a codebase task, an artifact might be a "list of identified functions"; for a long document, an "extracted event timeline." These are structured work products—more precise than plain-text summaries.
The raw predecessor trajectory is stored separately and not passed by default to the next round. Only when the next round finds it needs to look back does it retrieve it. This keeps the context from drowning in verbose history.
A Concrete Example
Suppose the task is: given a 200-page earnings call transcript, find every statement management made about AI investment, and judge which are commitments and which are outlook.
Traditional approach: stuff all 200 pages into context and answer in one shot. The result likely misses half the statements or mixes up different quarters.
Chained RLM approach:
The key difference: every root call is "fresh"—the model doesn't fatigue from reading 150 pages, and its attention isn't diluted by having already processed too much. Each time it restarts with concise work notes.
Experimental Results
The researchers compared Chained RLM against a standard LLM on four long-context benchmarks (both using GPT-5-mini):
| Benchmark | Standard LLM | Chained RLM | Gain | |------|---------|-------------|------| | RULER | 87% | 92% | +5pp | | BABILong | 44% | 59% | +15pp | | LongBench v2 | 41% | 52% | +11pp | | OOLONG-real | 14% | 38% | +24pp |
A clear pattern: the more a task requires aggregating information across long distances, the bigger Chained RLM's advantage.
The gain on RULER is small (+5pp) because many RULER tasks can be solved with local retrieval. The gain on OOLONG-real is largest (+24pp) because that task requires the model to "retain partial evidence, compare distant events, and aggregate many small observations"—exactly where single-pass inference fails most.
The Cost: More Expensive, But More Auditable
Chained RLM is not free. Its token consumption is about 1.8x that of a standard LLM, with costs scaling accordingly. The researchers ran the numbers (based on GPT-5-mini pricing):
Why This Matters
First, it shifts the "long context" problem from "stuff more" to "split smarter."
For the past two years, long-context research has been about "expanding windows from 32K to 128K, 1M, 10M." But Chained RLM points out: even with an infinite window, single-pass inference still rots. The problem isn't "can it fit," but "can it be processed well once it fits." This is a layer switch—from scaling up to divide and conquer.
Second, it echoes the principle of "granularity isomorphism."
The granularity of agent system optimization should match the granularity of the object being optimized. The natural granularity of long-document processing is "paragraph-level evidence integration," not "whole document in one pass." Chained RLM refines the reasoning unit from "one complete generation" to "one root call processing one segment," aligning exactly with the task's natural granularity.
Third, it makes intermediate reasoning auditable.
Traditional one-shot inference is a black box—the model reads 200 pages and gives an answer, and you don't know how it reasoned in between. Every Chained RLM root call leaves behind a summary, blackboard, and artifacts. You can inspect how the model understood the first 100 pages at root call 2, and roll back to root call 2's results and re-run root call 3. This is another instance of "division of labor beats unification" applied to inference architecture—not making one model do everything at once, but letting the same model work in shifts and do one thing right each time.
Honest Limitations
The researchers acknowledge: Chained RLM's gains on RULER are limited, showing that for tasks solvable by direct retrieval, the overhead of divide-and-conquer isn't worth it. And the doubled cost is real—not every scenario justifies paying twice as much for auditability and accuracy. The authors also didn't test on all long-context benchmarks; OOLONG-real's 14% → 38% is a big relative gain, but the absolute number is still not high.
Yet this work's value isn't "how much better than single-pass inference"—it's that it points to a path different from "keep expanding the window": make reasoning itself an engineering process that can be divided, handed off, and audited.
---
Paper: Chained Recursive Language Models for Multi-Iteration Reasoning, Mitra & Ulukus, 2026
Code: alexzhang13/rlm (GitHub)