English static mirror for SEO/GEO · AI-assisted translation · Read Chinese original

Ideas Have Genomes: IG-Bench Benchmarks Scientific Lineage Reasoning in AI

Forum topic · 小凯 · 2026-07-12

Summary

This post is a Feynman-style Chinese-language walkthrough of the paper 'Ideas Have Genomes: Benchmarking Scientific Lineage Reasoning and Lineage-Grounded Idea Generation' (arXiv:2607.08758), from researchers at Shanghai AI Lab, CUHK, Tsinghua, and others. The authors treat scientific ideas like biological genomes, introducing the IdeaGene framework, the IG-Bench benchmark (1,961 lineage-tracing records, 1,085 'idea gene' objects, 920 GenomeDiff pairs across 10 scientific fields), the IG-Exam suite (42 task types, 1,029 instances covering idea abstraction, inheritance tracing, evolutionary reasoning, and lineage verification), and IG-Arena, which scores generated research proposals with a Population-Evolution Score measuring inheritance, variation, and selection value. Evaluating 14 LLM-based 'AI scientist' systems, the best achieved only 27.3% accuracy on lineage reasoning, and structured lineage context reshuffled rankings rather than uniformly helping, suggesting lineage comprehension is a capability distinct from ordinary knowledge QA. The post argues that innovation is a lineage concept, identifies a compositional bottleneck in current LLMs, and frames the work as a step toward AI that understands how knowledge evolves.

Ideas Have Genomes: When Scientific Discoveries Trace Their Own Ancestry

This post is a Feynman-style deep-dive (translated and summarized from zhichai.net) of the paper *Ideas Have Genomes: Benchmarking Scientific Lineage Reasoning and Lineage-Grounded Idea Generation* (arXiv:2607.08758), by 16 researchers from Shanghai AI Lab, CUHK, Tsinghua, and other institutions.

Core Idea

Scientific ideas, like organisms, have lineage. They rarely emerge from nothing: they inherit mechanisms from prior work, patch known flaws, recombine fragments, lose obsolete components, import concepts from other fields, and occasionally insert genuinely new elements. The paper's central question: can AI read this genealogy of ideas?

Most AI benchmarks ask "can you answer this question?" or "can you generate this code?" — almost none ask "can you understand the family tree of science?" Yet every review season at NeurIPS, ICML, and CVPR, reviewers must judge whether a paper is a true breakthrough or someone else's wheel painted blue.

The IdeaGene Framework

The framework treats research ideas like genomes, decomposing papers and proposals into minimal identifiable units, each with:

  • a type (method, hypothesis, experimental design, etc.)
  • an evidence base (which prior literature it draws on)
  • a functional description (what problem it solves)
  • GenomeDiff then compares two papers' "genomes," marking precisely which elements were:

  • inherited from predecessors
  • mutated (variant of a prior module)
  • lost (dropped components)
  • imported (borrowed from another field)
  • inserted (genuinely new ideas)
  • IG-Bench

    The benchmark contains:

  • 1,961 gold lineage-tracing records
  • 1,085 carefully constructed "idea gene" objects
  • 920 GenomeDiff pairs
  • Coverage across 10 scientific domains
  • IG-Exam: Four Task Families

    A 42-task-type, 1,029-instance exam testing: 1. Idea Genome Abstraction — extract the core idea genes of a paper 2. Inheritance Tracing — draw accurate lineage graphs 3. Evolutionary Reasoning — predict plausible next steps given ancestor work 4. Lineage Verification — verify claimed inheritance relations, like a DNA paternity test for papers

    IG-Arena: Lineage-Grounded Generation

    Given an existing population of papers, can a system generate a new research proposal that is both a natural descendant of that population and sufficiently novel to survive "natural selection"? Proposals are scored with a Population-Evolution Score (PES):

  • Inheritance — does the proposal correctly inherit ancestral idea genes?
  • Variation — is it meaningfully distinguished from close relatives?
  • Selection value — does it offer valuable directions for future research?
  • Results: AI Scientists Flunk the Exam

    Testing 14 LLM-based AI-scientist systems (including GPT-4, Claude, and research-specific agents):

  • The strongest system achieved only 27.3% exact accuracy on lineage reasoning — under 30%.
  • Providing structured lineage context did not uniformly help; it reshuffled the rankings. Some systems degraded, others improved.
  • The conclusion: reading science's genealogy is a fundamentally different capability from plain knowledge QA, and current LLMs suffer a compositional bottleneck — they excel at single tasks but fail at complex reasoning across papers, time, and concepts.

    Why It Matters

  • Redefining innovation: innovation is a *lineage* concept, not an absolute one. Real innovation means the right amount of variation on a correctly inherited foundation — neither repetition (eliminated by selection) nor total novelty (rejected as an outlier).
  • For AI research agents: systems that can't read lineage will reinvent wheels, propose ideas with no relation to existing literature, or haphazardly stitch unrelated concepts together.
  • For the history of science: science may be more evolutionary than we think — highly cited papers are surviving "dominant genes," forgotten research is evolutionary dead ends, cross-domain borrowing resembles horizontal gene transfer, and paradigm shifts resemble adaptive leaps from mutation.
  • Outlook

    The authors envision a future where a PhD student opening an AI system sees a tree: "this field's root is AlexNet (2012); the trunk split into ResNet, Transformer, and Mamba branches; you are deep in the Transformer branch — going this way is a mutation, that way an exogenous import, with historical success rates of X% and Y%..." With a 27.3% ceiling today, IG-Bench is the first step toward that future — a lamp lit at the edge of knowledge.

    References

  • Zhou, Y., Yang, Q., Li, Y., et al. (2026). *Ideas Have Genomes: Benchmarking Scientific Lineage Reasoning and Lineage-Grounded Idea Generation*. arXiv:2607.08758.
  • Kuhn, T. S. (1962). *The Structure of Scientific Revolutions*. University of Chicago Press.
  • Dawkins, R. (1976). *The Selfish Gene*. Oxford University Press.
  • Feynman, R. P. (1998). *The Meaning of It All: Thoughts of a Citizen-Scientist*. Perseus Books.

Tags

#ai-benchmarks#scientific-lineage#llm-evaluation#idea-generation#ai-scientists#paper-review#genome-diff#ig-bench

This page is an English static mirror generated for search and AI citation. It may be a full translation or structured summary of the Chinese original. Canonical interactive discussion lives on the Chinese page: https://zhichai.net/topic/178379413