[论文] Re-Search: Dissecting LLM Ideation Through a Two-Player Questioning Ga...

论文概要

研究领域: ML
作者: Muhan Gao, Hangxiao Zhu, Xinqi Su, Yu Zhang, Daniel Khashabi, Kuan-Hao Huang
发布时间: 2026-11-10
arXiv: 2611.14487

中文摘要

研究构思是迭代的：研究者根据反馈一步步打磨想法。但现有评估掩盖了这一搜索过程，难以理解 LLM 如何探索研究想法空间、是什么限制了其进展。我们提出 Re-Search，把"二十个问题"适配到研究构思：只给一篇已发表论文的研究问题，模型必须通过是非问句重新发现论文的核心想法。基于 200 篇 ICML 2026 论文构建的 Re-Search 追踪模型想法如何随提问与二元神谕反馈演化，将构思与执行分离。六个近期 LLM 上，几乎所有进展都发生在 100 个问题中的前 30 个之内，这部分基本决定游戏结局；之后多数模型陷入平台期。通过对候选问题与假设空间的受控干预，发现瓶颈在生成而非判断：模型被提供时能认出好想法或好问题，但往往自己提不出来。对构思轨迹的进一步分析显示，失败方式要么是过度利用单一方向，要么是过度探索过多方向。这些发现为自动研究系统指出具体设计原则：既然模型能判断自己提不出的候选，用外部来源的想法与问题丰富搜索，可能比提升单一模型的生成能力更有效。

原文摘要

Research ideation is iterative: researchers refine their ideas step by step based on feedback. Yet existing evaluations obscure this search process, making it difficult to understand how LLMs explore the space of research ideas and what limits their progress. We introduce Re-Search, which adapts the format of Twenty Questions to research ideation: given only the research problem of a published paper, a model must rediscover the paper's core idea by asking yes-or-no questions. Built from 200 ICML 2026 papers, Re-Search tracks how a model's idea evolves as it asks questions and receives binary oracle feedback, isolating ideation from execution. Across six recent LLMs, nearly all progress happens within the first 30 of 100 questions, which largely decide how the game ends; after that, most mo...

自动采集于 2026-11-12

#论文 #arXiv #ML #小凯