[论文] Training Reasoning Models to Generate Diverse Scientific Ideas

论文概要

研究领域: NLP 作者: Manya Wadhwa, Pranav Narayanan Venkit, Hiroaki Hayashi, Yu Li, Tuhin Chakrabarty, Chien-Sheng Wu 发布时间: 2026-10-15 arXiv: 2610.22324

中文摘要

研究流程的第一步之一是产生想法。LLM 越来越多地被用于自动化这一步，但大多数工作优化单个想法的质量，而忽视了多样性。我们研究基于文档的想法生成，专注于提高 LLM 生成想法的多样性并确保其高质量。我们提出使用集合级强化学习目标进行训练：在单次生成中，模型编写 K 张结构化想法卡片，集合被联合评分，每个想法的质量按其与先前想法的距离加权。我们训练的 8B 模型在我们无污染的基准上取得了比 Claude Opus 4.8 和 GPT-6 Astra 更高的创意效用。在一项四位专家的试点研究中，我们的想法集被评为与 Claude Opus 4.8 的同样有希望，且包含的独特想法略多。当编码智能体对想法进行概念验证实验时，我们模型的 success@k 与 Claude Opus 4.8 相当。

原文摘要

One of the first steps of a research pipeline is generating ideas. Large Language Models (LLMs) are increasingly used to automate this step, but most of this work optimizes the quality of individual ideas, while overlooking diversity. In our work, we study document-grounded ideation, focusing on increasing the diversity of LLM generated ideas and ensuring they are high quality. Given prior papers, a model must propose a set of ideas that are each high-quality and clearly distinct from one another, while also being grounded in the input documents. To optimize for this, we propose training with a set-level reinforcement learning objective. In our setup, in a single generation, the model writes K structured idea cards grounded in the source papers, and the set is scored jointly, where each id...

#论文 #arXiv #NLP #小凯