Paper Overview
Research Area: Machine Learning Authors: Yifan Zhou, Qihao Yang, Yan Li Released: 2026-07-11 arXiv: 2607.08758
Abstract
Scientific ideas rarely start from a blank page. They inherit mechanisms, repair known limitations, and recombine pieces of earlier work, much like biological genomes. Current benchmarks still say little about whether AI systems can follow this inheritance structure. We present IdeaGene-Bench (IG-Bench), a benchmark for scientific lineage reasoning and lineage-grounded idea generation.
IG-Bench is organized around the IdeaGene framework: each paper or proposal is represented as a set of minimal, typed, evidence-grounded Idea Genome objects, and a GenomeDiff aligns these objects to record inheritance, mutation, loss, external import, and novel insertion under six operational evolutionary dynamics.
Benchmark Composition
- 1,961 golden lineage traces
- 1,085 curated Idea Genome objects
- 920 GenomeDiff record pairs
- Covers 10 scientific domains
- Idea Genome abstraction
- Inheritance tracing
- Evolutionary reasoning
- Lineage verification
- The strongest system achieved only 27.3% exact accuracy on lineage reasoning
- Structured lineage context reordered system rankings rather than uniformly helping every participant
- arXiv: https://arxiv.org/abs/2607.08758
Two Evaluations
IG-Exam
Closed-form lineage reasoning with 42 task types and 1,029 instances, covering:IG-Arena
Evaluates generation capability using lineage-grounded Population Evolution Scoring (PES). A proposal is assessed on whether it can be inserted as a coherent descendant of a given lineage population: it should inherit the correct Idea Genome objects, mutate meaningfully relative to neighboring work, and provide selection value for future research.Key Findings
Experiments on 14 LLM-based scientist systems exposed a compositional bottleneck: