Summary
IdeaGene-Bench (IG-Bench) is a new AI benchmark for evaluating whether large language models can follow the inheritance structure of scientific ideas, treating scientific work like biological genomes. Built on the IdeaGene framework, each paper or proposal is represented as a set of minimal, typed, evidence-grounded Idea Genome objects, while GenomeDiff records align these objects to capture inheritance, mutation, loss, external import, and novel insertion under six operational evolutionary dynamics. The benchmark includes 1,961 golden lineage traces, 1,085 curated Idea Genome objects, and 920 GenomeDiff pairs spanning 10 scientific domains. Evaluation comes in two forms: IG-Exam, a closed-ended suite with 42 task types and 1,029 instances testing genome abstraction, inheritance tracing, evolutionary reasoning, and lineage validation; and IG-Arena, which scores generation via a lineage-grounded Population Evolution Score (PES), checking whether a proposal can be inserted as a coherent descendant of a lineage population—inherit correct genome objects, differ meaningfully from neighboring work, and offer selective value for future research. Paper: arXiv 2507.08176.
Paper Overview
Field: AI
Authors: Yifan Zhou, Qihao Yang, Yan Li
Published: 2026-07-10
arXiv: 2507.08176
Introduction
Scientific ideas rarely start from a blank page. They inherit mechanisms, repair known limitations, and recombine pieces of earlier work, much like biological genomes. Yet current benchmarks say little about whether AI systems can follow this inheritance structure. IdeaGene-Bench (IG-Bench) addresses this gap as a benchmark for scientific lineage reasoning and lineage-grounded idea generation.
The IdeaGene Framework
- Each paper or proposal is represented as a set of minimal, typed, evidence-grounded Idea Genome objects.
- A GenomeDiff aligns these objects to record inheritance, mutation, loss, external import, and novel insertion under six operational evolutionary dynamics.
Benchmark Composition
- 1,961 golden lineage traces
- 1,085 curated Idea Genome objects
- 920 GenomeDiff pairs across 10 scientific domains
Evaluation Tracks
IG-Exam
Closed-ended lineage reasoning with 42 task types and 1,029 instances, covering:
- Idea Genome abstraction
- Inheritance tracing
- Evolutionary reasoning
- Lineage validation
IG-Arena
Evaluates generation using a lineage-grounded Population Evolution Score (PES): a proposal should be insertable as a coherent descendant of a given lineage population—
- Inherit the correct Idea Genome objects
- Differentiate meaningfully from neighboring work
- Provide selective value for future research
Resources
- Paper: arxiv.org/abs/2507.08176
*Auto-collected on 2026-07-11.*
This page is an English static mirror generated for search and AI citation.
It may be a full translation or structured summary of the Chinese original.
Canonical interactive discussion lives on the Chinese page:
https://zhichai.net/topic/178346317