Overview
Field: NLP Authors: Ziang Liu, James J. Kim, Yijia Dai, Jennifer J. Sun Published: 2026-11-10 arXiv: 2611.14514
Abstract
LLM agents are increasingly used for open-ended, data-driven scientific discovery, yet most of them evaluate hypotheses pointwise, testing an empirical claim \(h\) only against its complement \(\neg h\). When evidence rejects \(h\), the agent learns that one explanation is unlikely but not which alternative the evidence supports.
To recover that alternative, the authors introduce Contrastive Discovery, which replaces the single hypothesis with a contrast set — a scientific question paired with mutually exclusive and collectively exhaustive competing explanations. Each experiment is scored by how its evidence redistributes belief across the set, and the question is resolved when belief concentrates on one explanation. Because contrast sets change only the unit an agent proposes and scores, an existing agent can use them to re-evaluate already collected evidence or to guide search.
Key Results
- Across six domains, 89% of pointwise candidates with the largest belief change were rejections; with the same evidence, contrast sets resolved 73% of those to a specific alternative.
- Contrast-guided search solved 67% more questions than pointwise surprise under the same experiment budget.
- In blinded evaluation, 38 human experts selected the same explanation in 78% of solved questions, and 82% on questions where a pointwise agent rejected and stopped.
- Explicitly competing explanations enable discovery agents to solve more scientific questions from the same evidence and budget.
*Auto-collected on 2026-11-12.*