Is Capability a Liability? Why Stronger AI Models Make Worse Forecasts Before Major Disruptions

Forum topic · QianXun · 2026-05-25

Summary

A May 2026 paper from UC Berkeley and the Forecasting Research Institute, titled 'Is Capability a Liability? More Capable Language Models Make Worse Forecasts When It Matters Most,' reveals an inverse scaling effect in AI forecasting. Stronger models like Llama-3.1 405B detect superlinear (exponential) growth trends earlier than smaller ones, but this superior pattern recognition leads to 'aggressive extrapolation': they push rising curves far upward and inflate the upper tail of their predicted distributions with excessive confidence. When real-world regime changes occur—such as lockdowns ending an epidemic or policy shifts crashing markets—these inflated upper bounds become large errors. Experiments show stronger models remain accurate under linear growth, but under exponential growth combined with trend reversal, their CRPS scores deteriorate sharply, sometimes performing worse than smaller models. The mechanism behind when models stop extrapolating remains a black box: it may stem from overlearning growth narratives in pretraining data or RLHF rewards favoring confident answers. The finding challenges the assumption that bigger models are always better, warning that highly capable AI may be the last to brake during sudden turns in high-stakes forecasting of epidemics, financial bubbles, and runaway inflation.

Why More Capable AI Models Make Worse Forecasts When It Matters Most

This post discusses a May 2026 paper from researchers at UC Berkeley and the Forecasting Research Institute (FRI):

"Is Capability a Liability? More Capable Language Models Make Worse Forecasts When It Matters Most"

Authors: Nick Merrill, Jaeho Lee, Ezra Karger (UC Berkeley, Forecasting Research Institute, et al.)
arXiv ID: 2605.15840
Fields: Forecasting science, AI risk assessment, scaling laws
Keywords: Inverse scaling, superlinear growth, aggressive extrapolation, regime change

The Core Idea: A Weather Forecaster Analogy

Imagine two weather forecasters. The novice only says "maybe bring an umbrella" when it rains. The genius can compute every cloud's thickness and tell you the rain will last exactly 142 minutes, with exponential intensity.

Normally you'd pick the genius. But when a once-in-a-century drought hits, the genius—committed to his perfect "rainfall explosion model"—insists rain will be catastrophic and calculates flooding. The novice, meanwhile, scratches his head and says: "Hmm, the sky changed. The rain might not come."

This paper confirms that phenomenon in AI: models that understand patterns too well can be led astray by those very patterns, precisely when the stakes are highest—infectious disease outbreaks, financial bubbles, or runaway inflation.

Inverse Scaling and Aggressive Extrapolation

AI research generally assumes scaling laws: bigger models trained longer perform better. This paper identifies a monster called inverse scaling.

When data shows superlinear growth (e.g., exponential doubling early in an epidemic), stronger models (e.g., Llama-3.1 405B) detect the trend earlier and more sharply than weaker models (e.g., 70B). That should be a good thing. But because they are so good at pattern recognition, they exhibit aggressive extrapolation: they project the runaway curve straight into the sky—and they are overconfident in doing so.

Regime Change: The Logic Blind Spot

Why does this aggression cause failure? Because the real world exhibits regime change:

An epidemic can stop abruptly due to a government lockdown.
A market can crash instantly on a sudden policy shift.

Strong models, seeing an upward trend, raise the upper tail of their predicted distributions substantially—assuming a very high ceiling with high probability. When reality suddenly brakes or reverses, that inflated upper bound becomes a massive forecast error.

Experimental findings:

Under linear (moderate) growth, strong models remain stable: more capability means more accuracy.
Under exponential growth combined with trend reversal, strong models' CRPS scores fall off a cliff—sometimes performing worse than small models from years earlier.

The Unexplained "Extrapolation Black Box"

The paper diagnoses the condition but not the cure. A core open question remains:

How does the model internally decide when to stop extrapolating?

Does this aggressive tendency come from overlearning "growth mythology" or success narratives in pretraining data, or from RLHF rewarding humans' preference for confident answers? We know the model is "sick," but where exactly the surgical incision should go—how to teach AI situational caution—remains unsolved.

Takeaway

The end of wisdom is reverence for uncertainty.

This paper punctures blind faith that "bigger models solve everything." In high-stakes decisions, a capable AI may be a perfect driver on a smooth road—but the last to hit the brakes on a sharp turn.

Next time an AI predicts an industry will "grow forever" or a risk will "destroy the world," don't just marvel at its compute. It may simply be extending the straight line of "experience" into a hallucination called extrapolation, drawing it beyond the boundary of human civilization.

True foresight is seeing the ending behind the boom.