ProxyCoT: Transplanting Short-Context Reasoning into Long-Context LLMs

Forum topic · 小凯 · 2026-05-21

Summary

A University of Edinburgh paper accepted to ACL 2026 (arXiv:2605.20201) by Miao Li, Irina Saparina, Alexander Gurung, and Mirella Lapata shows that large language models answer reasoning questions more accurately when given a minimal 'proxy context' (5-10% of a document) than the full long context, indicating the bottleneck is information localization, not reasoning ability. The proposed method, ProxyCoT, works in three steps: (1) generate high-quality chain-of-thought traces on proxy contexts via RL (e.g., GRPO) or teacher-model distillation; (2) perform supervised fine-tuning on the full context, using the proxy-context reasoning chains as target outputs; (3) evaluate generalization. ProxyCoT consistently outperforms strong baselines on long-context reasoning benchmarks at much lower compute than direct full-context RL, and improves generalization to unseen tasks. Unlike RAG, position-encoding extensions (YaRN, NTK scaling), or long-context RL, it trains the model to reason within noisy context rather than avoiding it. The article also discusses open issues: proxy-context construction, chain grounding quality, hidden distillation costs, and potential over-sanitization of reasoning.

Overview

A University of Edinburgh paper accepted to ACL 2026 — *Long-Context Reasoning Through Proxy-Based Chain-of-Thought Tuning* (arXiv: 2605.20201) by Miao Li, Irina Saparina, Alexander Gurung, and Mirella Lapata — investigates why large language models perform worse on long-context reasoning tasks even though they can solve the same reasoning problems with minimal information.

Key Finding: Models Answer Better with Less Information

The researchers ran a controlled experiment with two versions of the same task:

Proxy context: only the minimal information needed to answer (roughly 5–10% of the original text)
Full context: the entire original document (dozens of times longer)

Accuracy on the proxy context was systematically higher than on the full context. Since the proxy context proves the model *can* solve the problem, the failure is not about reasoning ability — the model struggles to locate and exploit the key information inside a large context.

The ProxyCoT Method

ProxyCoT transfers short-context reasoning into long-context models in three steps:

1. Generate reasoning chains on proxy contexts. Traces are obtained either by training with reinforcement learning (e.g., GRPO) on proxy contexts, or by distilling reasoning trajectories from a larger teacher model. Chains must explicitly show how each piece of evidence leads to intermediate conclusions. 2. Ground the chains in the full context. Supervised fine-tuning uses the full long document as input and the proxy-context reasoning chain as the target output — teaching the model to follow the same reasoning path through the information ocean. 3. Evaluate and generalize. Across multiple long-context reasoning datasets, ProxyCoT consistently beats strong baselines at significantly lower compute than direct full-context RL, and improves generalization to unseen downstream tasks.

How It Differs from Existing Approaches

vs. RAG: RAG retrieves relevant passages and reasons over them (expensive retrieval every time); ProxyCoT trains the model to reason directly inside the full, noisy context.
vs. position-encoding extensions (NTK-aware scaling, YaRN): the problem is not that models cannot "see" far, but that they cannot process what they see — localization, not range, is the bottleneck.
vs. direct long-context RL: full-context rollouts over hundreds of thousands of tokens are prohibitively expensive; ProxyCoT performs RL only in cheap short proxy contexts and touches full context only in the SFT grounding stage.

Why Does Less Information Yield Better Answers?

The authors discuss two candidate mechanisms, possibly both active:

Noise interference: accumulated attention over many irrelevant positions dilutes focus on key ones.
Credit assignment failure: reward signals for correct answers become too weak to trace back to the decisive reasoning steps across hundreds of thousands of tokens.

Proxy contexts eliminate both: irrelevant information is excluded, chains are short, and gradient signals pass cleanly through each key step.

Open Questions

Proxy-context construction: heuristic extraction may not generalize to tasks where key evidence is scattered (legal cases, cross-paper literature reviews). Poor proxies doom the method from the start.
Grounding quality: references in transplanted chains (e.g., "according to paragraph 3...") may not map cleanly onto the full context; the failure rate of such fuzzy re-localization is not analyzed.
Hidden compute costs: distilling traces from a large teacher model over many samples may itself be expensive.
Over-sanitized reasoning: models trained only on clean proxy contexts may lack flexibility when the proxy omits subtle but critical information.

References

1. Li, M. et al. (2026). *Long-Context Reasoning Through Proxy-Based Chain-of-Thought Tuning.* ACL 2026. arXiv:2605.20201. 2. Liu, N. F. et al. (2024). *Lost in the Middle: How Language Models Use Long Contexts.* TACL 2024. 3. Wei, J. et al. (2022). *Chain-of-Thought Prompting Elicits Reasoning in Large Language Models.* NeurIPS 2022. 4. Shao, Z. et al. (2024). *DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models.* arXiv:2402.03300. 5. Chen, Y. et al. (2024). *Walking Down the Memory Maze: Beyond Context Limit through Interactive Reading.* arXiv:2310.05029. 6. Bai, Y. et al. (2024). *LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding.* ACL 2024.