AI Can Learn Scientific Taste: Teaching AI to Judge High-Impact Research via RLCF
> Paper: AI Can Learn Scientific Taste > Authors: Jingqi Tong, Mingzhe Li, Hangcheng Li, et al. (Fudan University + OpenMOSS team) > Code: https://github.com/tongjingqi/AI-Can-Learn-Scientific-Taste
Can Scientific Taste Be Taught?
Great scientists are often said to possess a mysterious "intuition"—the ability to foresee which directions are worth pursuing. This has long been considered an ineffable human trait called scientific taste. This paper argues otherwise: scientific taste is learnable, and AI has already learned it.
The Missing Piece in AI Scientists
Current AI-for-Science work focuses on literature retrieval and automated experimentation. But there is a critical gap: **AI scientists know *how* to do research, but not *what* to do. They can execute a research plan but cannot judge whether the plan is worth executing—distinguishing genuinely impactful ideas from those that are superficially novel but trivial.
RLCF: Turning Community Feedback into Supervision
Inspired by Hume (taste as the collective verdict of qualified judges) and Kant (taste as a sensus communis), the authors treat citations as the scientific community's long-term vote on a paper's value.
From Absolute Counts to Pairwise Preferences
Raw citation counts suffer from field bias and time bias. The solution: pairwise comparison. The SciJudgeBench dataset is built from 2.1 million arXiv papers, where each pair is same-field + same-year, with the more-cited paper labeled as better—yielding 700,000 preference pairs.
Training Pipeline (RLCF)
1. Community preference construction: extract pairwise signals from citations 2. Preference modeling — Scientific Judge: a generative reward model trained with GRPO; input is two papers' titles/abstracts, output is reasoning plus a verdict (reward: 1 if correct, 0 if not) 3. Preference alignment — Scientific Thinker: the judge serves as a reward model to train a policy model to propose potentially high-impact research directions via Comparison-Based GRPO
Results
Scientific Judge
- Data scaling (approx. log-linear): Qwen3-4B improves 60.3% → 75.3%; Qwen3-30B 66.3% → 80.6%
- Model scaling: Qwen2.5 series from 1.5B (72.1%) to 32B (83.7%); Qwen3-30B surpasses all listed closed-source models (e.g., GPT-5.2, Gemini 3 Pro)
- Temporal OOD: trained only on data through 2024, tested on 2025 papers—accuracy gains up to +55.1 pp
- Field OOD: trained only on CS, still improves predictions in Math, Physics, and other fields; also generalizes to bioRxiv biology papers
- Metric OOD: trained on citations, tested on peer-review scores (ICLR submissions)—gains up to +72.0 pp
- Citations are not random numbers but the sediment of long-term community interaction; noise averages out at scale
- Pairwise comparison removes confounders (field, year, citation scale)
- GRPO is extended to open-ended tasks without ground truth by casting correctness and pairwise preference as verifiable rewards
- Paper: https://arxiv.org/abs/2603.14473
- Code: https://github.com/tongjingqi/AI-Can-Learn-Scientific-Taste
- Dataset: SciJudgeBench (700K field- and time-matched citation preference pairs)
Scientific Thinker
Since there is no ground truth for proposed ideas, the authors use Comparison-Based GRPO: generate G candidate directions from a seed paper, run a round-robin tournament judged by Scientific Judge, and use win rates as rewards.
Trained on 4,000 highly cited 2025 papers, the Thinker's proposals were judged more promising than baselines by GPT-5.2-high, GLM-5, and Gemini 3 Pro (majority vote), both in-domain and out-of-domain.
Why It Works
Limitations and Open Questions
1. Citation delay: revolutionary papers may be "sleeper hits" with delayed citations 2. Citation culture differences across fields persist within comparisons 3. Self-citation and network effects: models may learn "star effect" rather than quality 4. Novelty vs. impact: highly original but slow-cited work may be undervalued 5. Counterfactual evaluation: current evaluation is "models judging models," risking circular validation 6. Negative impact: distinguishing praise-citations from criticism-citations remains unsolved
Conclusion
The paper's key contribution is a paradigm shift: scientific taste is not a mystical human trait but a transferable pattern learnable from community feedback. Potential applications include AI-assisted peer review, research direction recommendation, cross-domain knowledge transfer, and training data curation.
> As Hume argued that taste's standard comes from the collective verdict of qualified judges, scientific taste's standard comes from the community's collective citations. AI doesn't need to be Einstein to learn taste—it only needs to learn to read the community's collective wisdom.
References**