1. Introduction: The 'Emperor's New Clothes' of Long Context
1.1 The Symptom: GPT-4 as a 'Repeater' in Financial-Report Analysis
Vendors now ship LLMs with million-token context windows and market them as capable of handling unprecedented volumes of information. In practice, these models often behave as confused 'repeaters.' Feed a several-hundred-page financial report into a top model such as GPT-4 and it will at best echo surface figures or summarize individual sections. Ask for cross-chapter, multi-year reasoning such as 'compare revenue growth versus R&D investment across regional markets over the past three years and predict next-quarter risks' and performance collapses: logic becomes incoherent, statements contradict each other, or the model simply refuses to answer. The model can *see* more information, but cannot *understand* and *use* it for deep reasoning.
1.2 The Core Issue: Long Window ≠ Strong Reasoning
Behind this 'repeater' behavior lies what the industry calls Context Rot: as the context window grows to accommodate more tokens, reasoning ability does not scale with it and can even degrade. MIT researchers found that when input length and task complexity increase together, even frontier models like GPT-5 show cliff-like performance drops. Merely enlarging the window solves the 'can it fit' problem but not the 'can it think' problem. Transformer self-attention faces information dilution and positional-encoding failure at extreme lengths, making it hard to maintain coherent, precise reasoning across long documents. The real challenge is not how much a model can see, but how to make it filter, organize, and reason over huge information like a human expert.
1.3 MIT CSAIL's Disruptive Proposal: Recursive Language Models (RLM)
To address Context Rot, MIT CSAIL introduced Recursive Language Models (RLM). Instead of asking the model to passively memorize everything at once, RLM turns the LLM into an active manager that offloads work to an external environment, much as an operating system manages memory. RLM attaches a persistent Python REPL to the model, letting it write and execute code, and recursively delegate subtasks to smaller, cheaper sub-models. The long text is stored as a large data variable in the REPL rather than inside the context window. The model inspects, slices, searches, and filters this data via code, then recursively calls sub-models on individual segments. This divide-and-conquer strategy sidesteps the Transformer context-window ceiling and reframes long-document processing from a pure *memorization* problem into a more scalable *program-synthesis* problem.
2. Core Problem: 'Context Rot' — Transformer's Fatal Weakness
2.1 What Is 'Context Rot'?
#### 2.1.1 Definition: Performance Degrades as Input Grows
Context Rot describes a systematic, sometimes cliff-like, decline in LLM performance (especially on deep-reasoning tasks) as input context length grows. It is not simple forgetting but a systemic failure of the internal reasoning mechanism. Even when the context window technically fits the document, once length crosses a threshold the model loses its way and cannot integrate the information. 'Needle-in-a-haystack' retrieval may still work, but multi-step reasoning, aggregation, and cross-document association break down. The degradation is universal and worsens with task complexity, exposing the architecture's intrinsic limits on long-range dependencies.
#### 2.1.2 Symptom: 'Dementia' Even Inside the Window
Even when a long document fits the window, models exhibit several 'dementia' symptoms: information-extraction errors (mixing details across paragraphs or dropping key facts), broken logical chains (losing premises during multi-step reasoning), and inability to perform global analysis (treating sections in isolation rather than weaving them into an integrated conclusion). These symptoms show that simply enlarging the window does not grant true long-context understanding; instead, the self-attention mechanism gets diluted by irrelevant information, weakening the model's 'thinking.'
2.2 Why Does the Transformer Architecture 'Rot'?
#### 2.2.1 Attention Dilution: Information Loss in Long Sequences
Self-attention allows every token to attend to every other token. At hundreds of thousands or millions of tokens, each token's attention weights are spread across so many others that the true signal drowns in noise. As in a crowded stadium, individual voices become hard to isolate. Attention weights flatten and lose focus, destroying the model's ability to form long-range dependencies.
#### 2.2.2 Positional-Encoding Limits: Ineffective for Very Long Sequences
Positional encodings supply order information that self-attention lacks, but most schemes (absolute or relative) assume a fixed maximum length. Beyond that, new tokens either get no encoding or get a confused one. Even with flexible encodings such as RoPE, precision degrades as length grows, making it hard to distinguish far-apart but content-similar tokens. Loss of positional structure further damages long-text reasoning.
#### 2.2.3 'Phase Transition': Collapse from Memory to Reasoning
MIT researchers observed a sharper phenomenon: a phase transition in which performance does not degrade linearly but collapses suddenly once input length and task complexity cross a critical threshold. On simple retrieval the model behaves like a 'memorizer'; once the task demands comparison, aggregation, or ordering across many points, performance falls from a reasonable level to near-random. The switch from 'simple memory' to 'complex reasoning' has a fragile ceiling, and that ceiling drops sharply with input length—explaining why models suddenly become 'demented' on complex tasks such as financial-report analysis.
3. The Disruptive Solution: Recursive Language Models (RLM) — From 'Memorizer' to 'Manager'
3.1 Core Idea: Outsource Tasks Like an Operating System
#### 3.1.1 Analogy: A Smart Reporter Managing Mountains of Material
Imagine a reporter writing a deep piece on climate-change impacts on agriculture with thousands of reports, government documents, and press releases at hand. A clumsy reporter tries to read everything at once and write from memory—chaos results. A smart reporter builds a library (the Python REPL), indexes it, uses keyword search (REPL code) to locate relevant reports on sub-topics (e.g., 'drought effects on maize yield in Africa'), delegates close reading and summarization to assistants (sub-models via llm_query), and finally integrates, analyzes, and editorializes. RLM applies this divide-and-conquer wisdom, converting the LLM from a clumsy reporter into a resourceful one.
#### 3.1.2 Core Concept: Treat Long Text as an External Environment
RLM redefines the relationship between LLM and context. In the traditional paradigm, context is 'food' the model must swallow into its window. In RLM, long text is an external, manipulable environment and the model is an agent acting within it. The environment can be a Python REPL, a database, or a filesystem. Instead of 'seeing' everything, the model writes Python code to read a huge file into a variable, then uses string operations, regex, and slicing to inspect, filter, and search it. Because the model only ever consumes the (small) output of code execution, its window never overflows, enabling structured, controllable reasoning over arbitrarily large corpora.
3.2 Architecture: Python REPL and Recursive Calls
#### 3.2.1 Python REPL: Programming Power for the Model
The core of RLM is a persistent Python REPL (similar to a Jupyter notebook). When a user query arrives, the long context is loaded into this REPL as a Python variable (e.g., context), not into the model's context window. The model interacts with the REPL by emitting Python code blocks wrapped in markers such as ```repl. Example operations include:
- Peeking:
print(context[:1000])to inspect the first 1000 characters. - Searching:
import re; re.findall(r'pattern', context)to locate patterns. - Partitioning:
chunks = context.split('Chapter')to split by delimiters. - Storing:
important_data = context[5000:10000]to retain a filtered slice. - Performance edge: Full RLM outperforms all other methods on nearly every task. On BrowseComp+, RLM's accuracy (91.33%) exceeds the summary agent (70.47%) by over 21 points. On OOLONG-Pairs, RLM achieves a breakthrough from near-zero to 58%.
- Cost edge: RLM is highly competitive on cost. On CodeQA and OOLONG-Pairs, RLM has the lowest average cost among all methods. On BrowseComp+, RLM ($0.99) is well below the theoretical cost of calling GPT-5 directly ($1.50–$2.75).
- Combined value: RLM delivers superior accuracy and lower cost simultaneously. Smart decomposition plus selective processing means 'spending less to do more'—an attractive proposition for financial analysis, legal research, scientific literature review, and any scenario that demands deep reasoning over massive text.
The REPL executes the code and returns (typically truncated) output, giving the model an active, programmer-like grip on the data.
#### 3.2.2 Sub-Model Calls: Enabling Divide-and-Conquer
Beyond standard Python, the REPL exposes a special function, e.g., llm_query(prompt, sub_context), that launches a recursive sub-model call. This function hands a smaller, well-defined subtask to another LLM (same or smaller/cheaper). For example, after filtering ten relevant paragraphs from a long document, the root model passes each to llm_query with the instruction 'summarize the core argument of this paragraph.' Sub-models handle small contexts efficiently and return summaries for the root model to integrate. Complex tasks become a stream of manageable subtasks, processed in parallel or sequentially.
#### 3.2.3 Recursion: Decomposing Complex Tasks
Recursion is the soul of RLM. When a sub-model is invoked via llm_query, it can itself run as an RLM. If its subtask is still too complex, it can call llm_query again, decomposing the problem further. The process recurses until every leaf subtask is simple enough for a basic LLM. For example, a sub-model analyzing document A may discover it needs a fact from document B and can spawn another sub-model to find it. This produces a dynamic, adaptive task-decomposition tree whose depth and granularity match the actual need, enabling RLM to handle the deeply chained, multi-hop reasoning at which traditional LLMs are weakest.
3.3 RLM Workflow
#### 3.3.1 Loading Context into the Python Environment
The workflow begins by loading the long context into an isolated Python REPL along with metadata (total length, file type, etc.) that is included in the system prompt. For instance, the prompt tells the root LM: 'You are working in a REPL environment; the variable context contains a string of length 5,000,000 characters.' This primes the model to adopt a strategic, divide-and-conquer strategy instead of trying to ingest everything. The REPL is persistent, so variables and intermediate results survive across the entire query.
#### 3.3.2 Writing Code to Filter and Decompose
Equipped with this overview, the root model emits Python code to perform initial filtering and decomposition—the most strategic step, since quality depends on the model's decisions. In a 'needle-in-a-haystack' task it may use regex to focus on relevant keywords, shrinking the search space from millions of lines to dozens. For chapter-targeted analysis it may locate chapter headings and slice the target chapter out. Active code-based filtering is RLM's defining capability: it pulls exactly the relevant pieces from a sea of text, avoiding wasted compute and attention.
#### 3.3.3 Recursive Calls to Sub-Models
When filtered segments remain too large or too complex, the root model invokes the recursive call mechanism, passing one or more segments as sub-context along with a clear instruction to a sub-model. For example, it may send a 10,000-word chapter to a sub-model with 'summarize this chapter in under 200 words.' Sub-models work independently in their own small windows and return results. The root model can dispatch many sub-models in parallel or chain them based on earlier outputs, yielding a modular, scalable solution.
#### 3.3.4 Aggregating Results and Producing the Final Answer
Once sub-models return their 'outsourced' results, the root model collects them into REPL variables, may write further code to analyze or aggregate them, and may even invoke llm_query again for higher-level synthesis. When ready, it produces the final answer wrapped in a marker such as FINAL(answer) or FINAL_VAR(variable_name), signaling the end of reasoning. To the user it looks like one ordinary model call; behind the scenes, RLM has executed a recursive, multi-step reasoning pipeline.
4. Performance Validation: RLM on the Brutal OOLONG Benchmark
4.1 OOLONG: A Benchmark Designed for Long-Text Reasoning
MIT needed benchmarks that genuinely stress long-text reasoning. Many existing ones (such as 'needle-in-a-haystack') test only retrieval—an isolated fact in a huge context—which does not reflect complex reasoning. They adopted OOLONG, designed to evaluate reasoning and aggregation over long text. OOLONG requires not just finding information but also semantic analysis, transformation, and aggregation across many fragments, closer to the challenges of real financial reports, legal documents, and scientific literature.
#### 4.1.1 OOLONG-Pairs: A Quadratic-Complexity Challenge
MIT further designed OOLONG-Pairs, with quadratic complexity O(N²): the model must reason over every pair of entries. Given a large dataset of user interactions, the model must find all user pairs satisfying a condition—e.g., 'list all user pairs (no duplicates, smaller ID first) where both users have at least one instance tagged as "entity" or "human".' This demands pairwise comparison of all users. With N users, ~N²/2 comparisons are required. The resulting explosion of computation and memory defeats any model that tries to ingest everything at once.
#### 4.1.2 Task Definition: Pairwise Reasoning Over Inputs
OOLONG-Pairs demands a 'double-loop' reasoning ability: understand the query semantics, identify the entities (users) and the comparison condition, then design an efficient traversal strategy. An effective model must:
1. Identify and extract all user IDs and their tagged instances. 2. Filter and index users likely to satisfy the condition. 3. Pairwise compare the filtered users. 4. Aggregate and output all matching pairs in the requested format.
This tests not only information extraction and aggregation but also strategic planning and computational efficiency under massive input size. Traditional LLMs typically fail due to window overflow or attention dilution.
4.2 RLM vs. GPT-5: A Crushing Performance Gap
#### 4.2.1 GPT-5's Collapse: F1 of Just 0.04%
On OOLONG-Pairs, even frontier GPT-5 with its huge context window exhibits total 'dementia.' MIT's experiments show F1 of only 0.04%—essentially random guessing—because the task demands pairwise semantic comparison and aggregation across a huge input. As length grows, the number of pairs explodes, overwhelming GPT-5's context capacity and attention mechanism. The model cannot retain precise memory of every user and attribute, and reasoning collapses entirely. This confirms that even the best LLMs suffer severe Context Rot on high-complexity long-text tasks, and window size alone is not the answer.
#### 4.2.2 RLM's Rise: F1 Soars to 58%
By contrast, RLM-GPT-5 reaches 58.00% F1 on the same task—a qualitative leap from near-zero. RLM's success stems from divide-and-conquer and recursive calls. On OOLONG-Pairs, RLM does not try to process everything at once. It writes Python to parse the data structure, extract user IDs and tags, then—for instance—filters users with the required tag and runs pairwise comparison via recursive llm_query calls. Every sub-model works on a tiny local context, dodging Context Rot. Final aggregation yields the correct answer.
#### 4.2.3 Ablation: The Critical Boost from Recursion
MIT ran ablations to isolate each component's contribution. A non-recursive RLM variant (REPL access but no llm_query) reached 43.93% F1 on OOLONG-Pairs—still far above GPT-5's 0.04% but below the full RLM. This confirms that merely externalizing context to a REPL is itself hugely valuable: code-based interaction lets the model handle inputs far beyond its native window. But the >14-point gap (58.00% vs. 43.93%) highlights recursion's critical role. On information-dense tasks like OOLONG-Pairs, a few code interactions are not enough. Recursion enables deep, multi-level decomposition—breaking a big problem into smaller ones and those into even smaller ones—until every leaf subtask is reliably solvable by a lightweight sub-model. This depth of decomposition lets RLM handle intricate, multi-hop reasoning chains that the non-recursive variant cannot.
4.3 Cost Analysis: RLM Is Not Just Stronger but Potentially Cheaper
#### 4.3.1 Selective Processing Cuts Token Consumption
Performance is not the only metric; cost matters, especially in compute-heavy LLM workloads. Intuitively, RLM's many calls and interactions should cost more than a single base-model call. Surprisingly, MIT shows RLM can be not just stronger but also cheaper, overturning the 'more capability means higher cost' assumption. The reason is selective processing: traditional approaches (full-prompt input or summary-agent compression) inevitably burn tokens on irrelevant content, while RLM's REPL acts like a database, fetching only the slices relevant to the current subtask. 'Filter first, read carefully later' drastically reduces wasted tokens.
#### 4.3.2 Cost Comparison with Traditional Long-Text Approaches
MIT compared RLM with base-model calls, summary agents, and CodeAct + BM25 retrieval agents.
| Method | CodeQA (23K–4.2M tokens) | BrowseComp+ (1K) (6M–11M tokens) | OOLONG (131K tokens) | OOLONG-Pairs (32K tokens) | | :--- | :--- | :--- | :--- | :--- | | Base Model | 20.00%* | 0.00%* | 44.00% (GPT-5) | <0.1% (GPT-5) | | Summary Agent | 58.00% ($1.31) | 70.47% ($0.57) | 46.00% ($0.13) | 0.01% ($0.13) | | RLM (no recursion) | 58.00% ($0.18) | 88.00% ($0.44) | 36.00% ($0.37) | 43.93% ($0.69) | | Full RLM | 62.00% ($0.11) | 91.33% ($0.99) | 56.50% ($0.43) | 58.00% ($0.33) |
*\* indicates the method could not run because the input exceeded the context limit (treated as 0). Costs are average API costs in USD with standard deviations in parentheses. Source: Table 1 in the paper.*
Key takeaways:
5. RLM's Potential and Application Scenarios
5.1 Financial-Report Analysis: From 'Repeater' to Intelligent Analyst
RLM offers a revolutionary cure for the 'repeater' problem in financial-report analysis. Traditional LLMs on hundreds of pages of annual reports can only extract and echo figures; cross-chapter, multi-year reasoning is beyond them. RLM behaves like an experienced financial analyst: in the REPL it uses keyword search and structured extraction to locate key sections such as 'MD&A,' 'Notes to Financial Statements,' and 'Segment Information'; recursively calls sub-models to analyze cash-flow trends and segment profitability; and finally integrates results for cross-validation, trend forecasting, and a deep report covering risk assessment, peer comparison, and outlook. AI shifts from a figure-repeater to a truly intelligent analyst capable of complex, forward-looking reasoning.
5.2 Code Understanding: Handling Massive Codebases
Maintaining super-large codebases is a perennial software-engineering challenge. RLM provides a new approach: load the codebase into the REPL, write code to analyze directory structure and module dependencies, generate a high-level architecture map, and recursively call sub-models to deeply analyze core modules, functions, side effects, and more. This capability is valuable for code review, vulnerability detection, automated testing, and code generation—for example, when modifying a feature, RLM locates all related files and analyzes potential ripple effects, boosting developer efficiency and code quality.
5.3 Long-Document Summarization: Cross-Document Aggregation and Reasoning
Traditional summarization typically handles single documents; cross-document aggregation and reasoning are out of reach. RLM's recursive mechanism enables deep multi-document understanding. For instance, to survey dozens of papers on a topic, RLM calls sub-models to summarize each paper's core ideas, then the root model integrates the summaries, identifies connections, controversies, and evolution, and produces a clear, comprehensive review. This is valuable for academic research, market research, and intelligence analysis, accelerating knowledge extraction and creation.
5.4 Other Domains: Law, Science, Finance
RLM has broad prospects wherever long-text analysis and reasoning matter. In law, it can analyze legal documents and case law, surface relevant precedents, and flag contradictions or risky clauses in contracts. In scientific research, it can extract key information from massive literature, surface new directions, and support complex data analysis or modeling. In finance, it can analyze market reports, news, and social-media data for sentiment and forecasting, and assess complex derivatives. In short, any scenario requiring deep extraction and complex reasoning over huge volumes of text is fertile ground for RLM.
6. Philosophical Implications: Neuro-Symbolic Systems and the Future of AGI
6.1 The Fusion of Neuro-Symbolic Systems
#### 6.1.1 Neural Networks: Intuition and Semantic Understanding
RLM's design embodies the neuro-symbolic fusion. Neural networks (LLMs) provide intuition and semantic understanding. Their NLP power enables rapid grasp of meaning, sentiment, and context. In RLM, both the root LM's strategic decisions (e.g., where to slice) and sub-LMs' fine-grained analysis (e.g., a paragraph's core idea) rely on this intuition. Pattern-recognition-based neural perception provides the '感性' (sensibility) foundation for the whole system.
#### 6.1.2 Symbolic Systems: Logic and Precise Control
The symbolic side of RLM—mainly the Python REPL and code execution—provides logic and precise control. When the LLM needs deterministic, exact operations (computing lengths, filtering by rules, numerical computation, structured aggregation), it defers to this symbolic layer. Code execution is unambiguous, supplying a rigorous logical skeleton. Symbolic control compensates for neural networks' weakness in precise logic and complex calculation, enabling reliable reasoning. RLM's success is a textbook fusion of LLM 'fuzzy' intuition with code 'exact' logic.
6.2 Is RLM the Right Path Toward AGI?
#### 6.2.1 From 'Black Box' to Interpretable Reasoning
RLM also takes a major step toward interpretability. Traditional LLMs are largely black boxes whose reasoning is hard to audit. RLM decomposes reasoning into explicit, executable code steps and recursive calls, making the 'thinking' process transparent and traceable. We can clearly see how the model decomposes problems, filters information, calls sub-models, and integrates results. This structured, traceable reasoning aids debugging and trust, and is a valuable property on the road to AGI, where interpretability is crucial.
#### 6.2.2 From 'Memory' to 'Thinking': A Paradigm Shift
RLM represents a deep paradigm shift from memory to thinking. Traditional LLMs rely on massive parameters and training data, answering through memorization and pattern matching. RLM behaves more like a real thinker: it actively explores, decomposes, reasons, and verifies. It learns to use tools (the REPL), manage resources (recursive sub-model calls), and plan tasks (strategic decisions). This active, dynamic, structured cognition is closer to human thinking. Many researchers believe AGI requires exactly this kind of neuro-symbolic fusion—neural learning plus symbolic reasoning. RLM provides strong empirical support and outlines a clear blueprint toward more advanced AI.
7. Conclusion: RLM Reshapes AI's Future
7.1 Summary: How RLM Solves Context Rot
In sum, RLM solves Context Rot through a disruptive paradigm shift. Instead of treating long text as a memorization burden to be swallowed at once, RLM externalizes it as a programmable environment. By granting models the ability to write and execute Python, RLM turns LLMs into managers that actively and strategically filter and decompose information. The core mechanism—recursive calls—lets the model break a huge, complex reasoning task into a stream of smaller, manageable subtasks delegated to sub-models. This divide-and-conquer strategy sidesteps Transformer bottlenecks such as attention dilution and reasoning collapse, lifting effective reasoning ability by orders of magnitude.
7.2 Outlook: The Infinite Possibilities of Recursive Intelligence
RLM is more than a technical breakthrough; it is a profound signpost for AI's future. The neuro-symbolic fusion it embodies, and the shift from memory to thinking, point toward stronger, more reliable, more human-like AGI. Looking ahead, recursive intelligence based on RLM will expand AI's reach in many domains: intelligent economic advisors that deeply analyze global financial markets, automated software engineers who understand and maintain millions of lines of code, AI scientists who discover new knowledge and hypotheses from massive scientific literature. The era of recursive intelligence will vastly extend AI's capability boundaries, reshape how we interact with information, knowledge, and intelligence, and ultimately reshape both AI and human society.