Paper: Tracing the Scientific Discovery Capacity of LLMs to Their Training Data

Paper Overview

Field: NLP Authors: Arda Uzunoglu, Mahler Revsine, Michael Schatz, Benjamin Van Durme, Mehrdad Farajtabar, Daniel Khashabi, Eric Nalisnick Published: 2026-11-02 arXiv: 2611.03617

Summary

Large language models show growing promise for scientific discovery, yet basic questions remain about what enables reliable discovery. How can we trace the scientific discovery capacity of LLMs to their training data? The authors study this question as one of missing probability mass and develop a theoretical account in which monofacts—facts observed only once during training—set the discovery capacity.

In materials science and regulatory genomics, the researchers directly manipulate monofact rates in training sets and find that more monofacts consistently yield more discoveries across two open-weight model families, sampling strategies, and reasoning budgets. Further experiments show the relationship cannot be explained by training-set diversity alone and can transfer across scientific domains when paired with in-domain data.

Key Findings

• Monofacts (facts seen exactly once during training) are theorized to determine a model's scientific discovery capacity.
• Directly increasing the monofact ratio in training sets leads to more discoveries in both materials science and regulatory genomics.
• The effect is robust across two open-weight model families, multiple sampling strategies, and varied reasoning budgets.
• Training-set diversity alone cannot explain the relationship.
• The benefit can transfer across scientific domains when combined with in-domain data.

The paper concludes that the monofact rate in training data is a governing factor for LLM-based scientific discovery.

--- *Auto-collected on 2026-11-04*