The Smarter the Model, the Deadlier the Forecast: Inside LLMs' Inverse Scaling Puzzle in Disaster Prediction

Forum topic · 小凯 · 2026-05-22

Summary

A 2026 paper from the Forecasting Research Institute and UC Berkeley (arXiv:2605.22672, Merrill, Lee & Karger) shows that more capable language models make worse probabilistic forecasts when data features superlinear growth followed by a regime change, such as pandemics, housing bubbles, and hyperinflation. Using ForecastBench-Sim (built on FreeCiv game simulations), controlled SIR epidemiological models, and real-world datasets (COVID-19, US measles 1928–1962, Case-Shiller housing, hyperinflation episodes), the authors find capability correlates with better short-horizon forecasts but significantly worse long-horizon ones (e.g., CRPS Spearman correlation flipping from +0.67 at horizon 1 to -0.42 at horizon 7). Errors concentrate in the upper tail (p90), as stronger models over-extrapolate growth trends. A 2x2 Llama 3.1 experiment shows scale and RLHF each independently worsen crash-scenario errors. Critically, Brier scores on identical outputs show positive scaling while CRPS shows inverse scaling—meaning single-threshold benchmarks mask the failure. Even when models correctly identify historical crises (46/48 hyperinflation probes), they fail to apply that knowledge to calibration. The paper urges tail-aware metrics, revised training objectives, and decoupling knowledge from overconfidence in forecasting tasks.

Is Capability a Liability? More Capable Models Make Worse Forecasts When It Matters Most

Paper: *Is Capability a Liability? More Capable Language Models Make Worse Forecasts When It Matters Most* Authors: Nick Merrill, Jaeho Lee, Ezra Karger (Forecasting Research Institute / UC Berkeley) arXiv: 2605.22672, May 21, 2026, cs.AI

Key points

Inverse scaling in forecasting: Across 29 models from 7 providers and 5 model families, stronger models (higher Epoch Capabilities Index) forecast better at short horizons but *worse* at longer horizons when the data show superlinear growth followed by a regime change (crashes, interventions, collapses).
FBSim benchmark: The authors built ForecastBench-Sim on FreeCiv game simulations—contamination-free, with paired binary (Brier) and quantile (CRPS) scoring. Capability–CRPS correlation flips from +0.67 (H1) to -0.42 (H7, 95% CI [-0.72, -0.02]).
The tail is the problem: Lower-tail (p10) accuracy stays flat across horizons; the upper tail (p90) flips from +0.78 to -0.57. Stronger models over-extrapolate growth trends, pushing p90 far too high when a crash arrives.
Controlled SIR experiments: On 50 simulated SIR epidemic curves (exponential rise, intervention, decline), CRPS shows inverse scaling: ρ = -0.62 (p < 0.001, N = 27 models). Identically structured *linear* growth + crash shows positive scaling (ρ = +0.61). Superlinear growth + collapse together are what trigger the failure.
Scale and RLHF both contribute: A 2×2 Llama 3.1 experiment (70B/405B, Base/Instruct) shows scale and post-training each independently amplify crash-scenario errors; CRPS inflation >10x rises from 41% (70B-Base) to 63% (405B-Instruct).
Real-world replication: Inverse scaling appears in COVID-19 (60 countries, ρ = -0.54), Case-Shiller housing around 2005 (ρ = -0.67, the strongest effect), 12 hyperinflation episodes (ρ = -0.59), and pre-vaccine US measles (long horizons ρ = -0.42). Influenza—a preregistered control with weaker overshoot—shows no effect.

The most disturbing findings

Knowledge doesn't fix calibration. When prompted with domain labels, results vary wildly: the inverse scaling reverses for COVID-19, weakens but persists for housing, and has *zero effect* for hyperinflation. When probed directly, models correctly identified 46 of 48 hyperinflation events—yet still pushed median forecasts up to 7 million times the actual outcome. The knowledge is retrievable; it simply isn't routed into the forecast.

Metrics hide the failure. On the exact same outputs, Brier scores show positive scaling (ρ = +0.45) while CRPS shows inverse scaling (ρ = -0.42). Single-threshold metrics don't cover the tail, where the worst errors live. All major LLM forecasting benchmarks (ForecastBench, KalshiBench, etc.) currently report only binary-threshold metrics.

Why it matters now

The authors note active research deploying LLMs for real-time infectious disease forecasting (Du et al., 2025, *Nature Computational Science*) while measles resurges globally. Miscalibrated upper tails in epidemiological forecasts mean mis-timed peaks—missed hospital resource deployment when outbreaks actually explode.

Recommended fixes

1. Metric reform: All LLM forecasting benchmarks should report at least one tail-covering score (CRPS, log score, or multi-threshold Brier) alongside binary metrics. 2. Training objectives: RLHF-style reward on point accuracy independently worsens crash-scene overcommitment; scaling won't auto-correct it. 3. Decouple capability from confidence: The knowledge–calibration gap suggests a structural "knowledge routing failure" inside models—the mechanism is left for future work.

References

1. Merrill, N., Lee, J., & Karger, E. (2026). Is Capability a Liability? *arXiv:2605.22672*. 2. McKenzie, I. R., et al. (2023). Inverse Scaling: When Bigger Isn't Better. *TMLR*. 3. Schaeffer, R., Miranda, B., & Koyejo, S. (2023). Are Emergent Abilities of LLMs a Mirage? *NeurIPS 2023*. 4. Karger, E., et al. (2025). ForecastBench. *ICLR 2025*. 5. Du, H., et al. (2025). Advancing Real-Time Infectious Disease Forecasting Using LLMs. *Nature Computational Science, 5(6), 467–480*.