Training Reasoning Models to Generate Diverse Scientific Ideas

Forum topic · 小凯 · 2026-10-17

Summary

This paper investigates document-grounded scientific ideation with large language models, addressing the common limitation that prior work optimizes the quality of individual ideas while overlooking diversity. The authors propose a set-level reinforcement learning objective: in a single generation, the model writes K structured idea cards grounded in source papers, and the whole set is scored jointly, with each idea's quality weighted by its distance from previous ideas to encourage distinctiveness. The trained 8B model achieves higher creative utility than Claude Opus 4.8 and GPT-6 Astra on a contamination-free benchmark. In a pilot study with four experts, its idea sets were rated as promising as Claude Opus 4.8's while containing slightly more unique ideas, and when coding agents ran proof-of-concept experiments, its success@k was comparable to Claude Opus 4.8. Paper link: arXiv 2610.22324.

Paper Overview

Field: NLP Authors: Manya Wadhwa, Pranav Narayanan Venkit, Hiroaki Hayashi, Yu Li, Tuhin Chakrabarty, Chien-Sheng Wu Published: 2026-10-15 arXiv: 2610.22324

Introduction

One of the first steps of a research pipeline is generating ideas. Large Language Models (LLMs) are increasingly used to automate this step, but most existing work optimizes the quality of individual ideas while overlooking diversity. This work studies document-grounded ideation, focusing on increasing the diversity of LLM-generated ideas while ensuring they remain high quality.

Method: Set-Level Reinforcement Learning

Given prior papers, the model must propose a set of ideas that are each high-quality, clearly distinct from one another, and grounded in the input documents.
The authors propose training with a set-level reinforcement learning objective.
In a single generation, the model writes K structured idea cards grounded in the source papers.
The set is scored jointly: each idea's quality is weighted by its distance from previously generated ideas, rewarding diversity.

Results

The trained 8B model achieves higher creative utility than Claude Opus 4.8 and GPT-6 Astra on a contamination-free benchmark.
In a pilot study with four experts, the model's idea sets were rated as equally promising as Claude Opus 4.8's, while containing slightly more unique ideas.
When coding agents conduct proof-of-concept experiments on the generated ideas, the model's success@k is comparable to Claude Opus 4.8.