[论文] LongZero: Enhancing Long-Context Reasoning in LLMs from Zero Data

论文概要

研究领域: NLP 作者: Zhennan Wan, Jianfei Chen 发布时间: 2026-10-21 arXiv: 2610.30132

中文摘要

强化学习（RL）是增强大语言模型长上下文推理能力的有效途径，但它需要大量长文本、多样化、有挑战性且可验证的训练数据。人工标注这类数据代价高昂，自然产生的数据又过于稀缺，而现有合成方法依赖外部语料库或更强的教师模型——随着被训练模型变得更强，这些资源反而更难获取。近期，零数据RL（模型从自己提出的任务中学习）取得了出色表现，但现有方法仅针对短上下文任务。为此我们提出 LongZero：一种让 LLM 在完全不依赖外部数据的情况下提升自身长上下文推理能力的方法。LongZero 先在紧凑的结构化形式中合成问题、答案及其支撑证据，然后再将证据单独扩展为长多文档上下文。它将自生成的多跳推理任务与多轮检索任务相结合，使用 GRPO 算法训练模型。在两种 LLM 和五个长上下文推理基准上的实验表明，LongZero 能够泛化到真实任务，并超越了基于外部语料合成数据的基线方法，为改进 LLM 长上下文推理提供了一条可行且有前景的路径。

原文摘要

Reinforcement learning (RL) is an effective way to strengthen the long-context reasoning of large language models (LLMs), but it needs training data that are long, diverse, challenging, and verifiable. Human annotation of such data is expensive, naturally occurring data are too scarce to sustain RL, and existing synthesis methods depend on external corpora or stronger teacher models, which become harder to obtain as the trained models get stronger. Recently, RL from zero data, in which a model learns from tasks it proposes itself, has achieved excellent performance, but existing methods target short-context tasks. Therefore, we propose LongZero, a method that lets an LLM improve its own long-context reasoning without any external data. Instead of writing questions and answers over existing...

#论文 #arXiv #NLP #小凯