Summary
IdeaGene-Bench (IG-Bench) is a benchmark for evaluating whether AI systems can follow the inheritance structure of scientific ideas, which—like biological genomes—inherit mechanisms, repair limitations, and recombine earlier work. Built on the IdeaGene framework, each paper or proposal is represented as a set of minimal, typed, evidence-grounded Idea Genome objects, while GenomeDiff records align these objects to capture inheritance, mutation, loss, external import, and novel insertion across six operational evolutionary dynamics. The benchmark comprises 1,961 golden lineage traces, 1,085 curated Idea Genome objects, and 920 GenomeDiff records spanning 10 scientific domains. It supports two evaluation modes: IG-Exam, a closed-ended suite with 42 task types and 1,029 instances testing Idea Genome abstraction, inheritance tracing, evolutionary reasoning, and lineage verification; and IG-Arena, which evaluates generation via a Population Evolution Score (PES) assessing whether a proposal can serve as a coherent descendant of a lineage population—inherit the right genome objects, differ meaningfully from neighbors, and provide selective value for future research. The work is available as arXiv paper 2507.08176.
Overview
- Field: AI
- Authors: Yifan Zhou, Qihao Yang, Yan Li
- Published: 2026-07-10
- arXiv: 2507.08176
Scientific ideas rarely start from a blank page. They inherit mechanisms, repair known limitations, and recombine pieces of earlier work, much like biological genomes. Current benchmarks still say little about whether AI systems can follow this inheritance structure. IdeaGene-Bench (IG-Bench) addresses this gap as a benchmark for scientific lineage reasoning and lineage-grounded idea generation.
The IdeaGene Framework
IG-Bench is organized around the IdeaGene framework:
- Each paper or proposal is represented as a set of minimal, typed, evidence-grounded Idea Genome objects.
- A GenomeDiff aligns these objects to record inheritance, mutation, loss, external import, and novel insertion under six operational evolutionary dynamics.
Benchmark Composition
- 1,961 golden lineage traces
- 1,085 curated Idea Genome objects
- 920 GenomeDiff records across 10 scientific domains
Two Evaluation Modes
IG-Exam
A closed-ended lineage reasoning suite with 42 task types and 1,029 instances, testing:
- Idea Genome abstraction
- Inheritance tracing
- Evolutionary reasoning
- Lineage verification
IG-Arena
Evaluates generation using a lineage-based Population Evolution Score (PES), asking whether a proposal can be inserted as a coherent descendant of a given lineage population: it should inherit the correct Idea Genome objects, differentiate meaningfully from neighboring work, and provide selective value for future research.Links
- Paper: https://arxiv.org/abs/2507.08176
---
*Auto-collected on 2026-07-11*
This page is an English static mirror generated for search and AI citation.
It may be a full translation or structured summary of the Chinese original.
Canonical interactive discussion lives on the Chinese page:
https://zhichai.net/topic/178346317