This post is a GEO-optimized version of the original topic on zhichai.net.
Paper: OptimismBench: Forecasting Bias and the Alignment Effect in Language Model Judgment Authors: Cho Seonglae, Koshiyama Adriano arXiv: 2607.26981
Key points
- The puzzle: Ask an LLM for P(startup succeeds) → 70%. Ask P(it fails) → 15%. 70 + 15 = 85. The missing 15 points are not rounding error — they reveal a directional bias that calibration metrics (ECE, Brier score) cannot catch, because those metrics cancel out signed errors.
- Ground-truth-free bias detection: OptimismBench's 'inverted pair' method compares the model's answers to two phrasings of the same scenario (P(success) vs P(failure)). If the model is unbiased, they sum to 100%. The deviation defines the Skew metric: Skew > 0 = optimism, Skew < 0 = pessimism. No ground truth required.
- Four tracks: calibration control (15 questions with real frequencies), probability estimation (main inverted-pair track), recommendation (judgment replaced by recommending), and salience (varying scenario salience).
- Interventions fail: narrative manipulation, perspective shifting, anchoring gradients, and self-debiasing were all tested — bias survived every intervention.
- 14 of 16 models (8 vendors) are optimistic; only 2 are pessimistic — both from Anthropic's frontier Claude line.
- Post-training determines bias sign: across 11 base-vs-chat pairs in 4 families, Qwen's post-training shifted models from pessimistic to optimistic; Llama's shifted from optimistic to pessimistic. Bias is a product of alignment training, not architecture or pretraining.
- Model identity > language: across 17 models and 6 languages, between-model variance was 4.7x between-language variance.
- Bias flips by task framing: in the recommendation track, a model optimistic in probability judgment could become pessimistic. Bias is a joint property of model × task framing, not a fixed trait.
- The family-direction conclusion rests mainly on the Qwen and Llama pairs; Gemma-2-2b and Mistral-Small-24B are single-pair pilots with limited statistical power.
- Cross-language evidence is unbalanced: 6 native-language prompts plus 4 English system prompts (DE/FR/HI/JA), a known confounder.
- Skew measures internal consistency, not agreement with human judgment — an 'unbiased' model is not necessarily an 'accurate' one.
- No matched human-subject baseline, despite citing Weinstein (1980).
Main findings
Why it matters
1. Structural blind spot of calibration metrics: ECE and Brier score are unsigned — a model can be well-calibrated on average while systematically overestimating positive outcomes. Directional bias hides in the *direction* of error. 2. Alignment is not neutral: RLHF/DPO shapes not just safety and helpfulness but probability tilt. A model trained to be 'more helpful' may systematically overestimate success rates — dangerous for risk assessment, medical decisions, and financial forecasting. 3. Downstream inheritance: the paper's closing line:
> "When alignment makes a model more helpful, it also tilts its probabilities; downstream pipelines inherit the tilt by default."
Any system using LLMs for probabilistic judgment should audit the model's directional bias rather than assume 'the model said 70%, so it's 70%.'
Cross-domain analogy: human optimism bias
The title's 'Optimism' echoes the classic human optimism bias (Weinstein, 1980): people systematically overestimate positive events. LLMs appear to have absorbed this human cognitive pattern — but with a key difference: human bias is universal, while LLM bias direction is set by post-training, meaning alignment could counter-calibrate it. This raises an open question: should alignment target 'unbiased' or 'human-like optimism'? For competing with humans in prediction markets, the latter may be more accurate; for risk assessment, unbiased is correct.
Limitations (acknowledged by the authors)
Commenter's take
The methodological elegance is reframing: instead of comparing predictions to reality, compare the model to *itself*, sidestepping the ground-truth dependency. The practical worry: we increasingly use LLMs as probability-judgment engines, but their bias carries 'family traits' — a risk system built on Claude vs. one built on Qwen may point in opposite directions. And there is a real tension between alignment and calibration: helpfulness rewards optimistic answers, accuracy punishes them. The tension may have no perfect solution, but at least it can now be measured.
Links: Paper · HTML version · Dataset: 3,870 test items, 10 languages, open-sourced.