[论文] ReContext: Recursive Evidence Replay as LLM Harness for Long-Context R...

论文概要

研究领域: ML
作者: Yanjun Zhao, Ruizhong Qiu, Tianxin Wei
发布时间: 2026-07-04
arXiv: 2507.00478

中文摘要

理解和推理长上下文已成为在真实应用中部署大语言模型（LLM）的关键要求。尽管最近的LLM支持越来越长的上下文窗口，但它们往往无法利用输入中已有的相关证据，揭示了上下文访问与有效上下文利用之间的差距。在本工作中，我们提出了递归证据重放作为长上下文推理的LLM工具（RECONTEXT），一种无需训练即可改善长上下文推理的推理方法。RECONTEXT使用模型内部相关性信号构建查询条件化的证据池，并在最终生成前重放它，同时保留完整的原始上下文。这种递归选择过程将证据组织与答案生成分离，无需训练、外部记忆或上下文剪枝。我们还提供了基于联想记忆的理论分析，将上下文表征为记忆存储，将问题表征为检索线索，将注意力表征为线索-痕迹关联，将重放表征为痕迹再激活。在八个128K上下文长度的长上下文数据集上的实验表明，RECONTEXT持续改善了Qwen3-4B、Qwen3-8B和Llama3-8B的证据利用，在三个骨干模型上均取得最佳平均排名。

原文摘要

Understanding and reasoning over long contexts has become a key requirement for deploying large language models (LLMs) in realistic applications. Although recent LLMs support increasingly long context windows, they often fail to use relevant evidence that is already present in the input, revealing a gap between context access and effective context utilization. In this work, we propose Recursive Evidence Replay as LLM Harness for Long-Context Reasoning (RECONTEXT), a training-free inference method for improving long-context reasoning. RECONTEXT uses model-internal relevance signals to construct a query-conditioned evidence pool and replays it before final generation while preserving the full original context. This recursive selection process separates evidence organization from answer gener...

自动采集于 2026-07-04

#论文 #arXiv #ML #小凯