Overview
This post discusses a research paper: *Prospective multi-pathogen disease forecasting using autonomous LLM-guided tree search* (arXiv:2605.16238), by Sarah Martinson, Michael P. Brenner, Martyna Plomecka, Brian P. Williams, Nicholas G. Reich, and Zahra Shamsi (Google DeepMind, Harvard, and collaborators).
The paper tackles the expert bottleneck in infectious disease forecasting. Traditionally, the CDC's forecast hubs aggregate models hand-built by dozens of research teams-an approach that is labor-intensive, slow to scale to new pathogens or finer geographic granularity, and infeasible to replicate quickly.
Key points
- Core idea: Model development is framed as a code optimization problem. An LLM proposes code improvements grounded in epidemiological theory or statistical methods, while a tree search systematically explores the code space, keeping only variants that improve a target metric (weighted interval score, WIS).
- Fully prospective evaluation: The system ran in real time during the 2025-2026 US respiratory season, producing forecasts without any access to future data-in contrast to common retrospective evaluations.
- Results: For influenza, COVID-19, and RSV, the system autonomously discovered methodologically diverse models whose ensemble matched or exceeded the CDC's human-curated hub ensemble in out-of-sample evaluation.
- Cold-start robustness: RSV, with limited historical data and less mature surveillance, served as a hard test case; the system still discovered effective models, demonstrating generalization beyond data-rich diseases.
- Guarding against reward hacking: Two safeguards emerged from controlled retrospective ablations: 1. Optimizing log-scale distance metrics, which forces order-of-magnitude-meaningful improvements rather than mathematically exploitative tweaks. 2. An automated judge-in-the-loop that checks whether generated code is structurally consistent with established scientific theory (e.g., conservation of population in SIR-type models), acting like automated peer review.
- Workforce bottleneck broken: Weeks-to-months of expert modeling compressed into days, enabling rapid models for emerging pathogens, county-level granularity, and lower barriers for regions with limited modeling capacity.
- Transparency: Unlike black-box neural networks, the generated code is readable and verifiable by human experts, addressing a major trust barrier for AI in public health.
- Generality: The same LLM-guided tree search approach has produced expert-level results in other domains, including single-cell RNA-seq analysis (reportedly discovering 40 new methods surpassing public leaderboard bests), geospatial analysis (U-Net + Transformer architectures), and zebrafish neural activity prediction- suggesting a general scientific discovery accelerator rather than a disease-specific trick.
- Quality depends on input data availability; the automated judge requires predefined structural constraints; tree search is computationally heavy (300-2000 nodes per search); initial configuration still requires human experts.
- Deeper questions raised include the evolving role of human scientists, accountability when AI-generated models inform public health decisions, and how to arbitrate between conflicting AI-generated forecasts.
- Martinson, S., Brenner, M. P., Plomecka, M., Williams, B. P., Reich, N. G., & Shamsi, Z. (2026). Prospective multi-pathogen disease forecasting using autonomous LLM-guided tree search. arXiv:2605.16238.
- Aygün, E., et al. (2025). An AI system to help scientists write expert-level empirical software. arXiv:2509.06503.
- CDC FluSight Forecasting: https://www.cdc.gov/flu/weekly/flusight/
Why it matters
Limitations and open questions
Conclusion
The post frames this not as AI replacing humans, but as AI expanding human capability: an autonomous system that translates epidemiological theory into transparent, executable forecasting software, removing the manual-software bottleneck on the scientific discovery loop.