MIST Benchmark and SCOPE: Teaching LLMs When to Trust Context

The Counterintuitive Dilemma

A model trained only to resist misleading inputs may stop trusting useful ones. Optimizing for robustness against manipulation can silently destroy the ability to accept correct context—a form of false robustness invisible to single-condition accuracy metrics.

The paper "Learning When to Trust via Selective Context Preference Optimization" (Xian Sun et al., 2026) makes this trade-off measurable and optimizable through a new benchmark (MIST) and a new training method (SCOPE).

MIST: Four Paired Conditions

MIST renders each question under four conditions:

1. Clean – question only. 2. Misleading – plausible but wrong extra context. 3. Correct-Context – extra context pointing to the right answer. 4. Irrelevant-Context – unrelated additional context.

This paired design isolates two independent abilities: resisting misleading signals and accepting useful information.

SC2W: A Paired Metric

$$\text{SC2W}=\frac{\sum_i \mathbb{I}[a_i^{\text{clean}}=1 \wedge a_i^{\text{mis}}=0]}{\sum_i \mathbb{I}[a_i^{\text{clean}}=1]}$$

In words: among questions the model answers correctly in the clean condition, what fraction flips to wrong under misleading context? SC2W isolates pure susceptibility rather than overall accuracy.

Universal Susceptibility

Large-scale evaluation shows no model truly "judges" context:

• GPT-5.5: 10.5%
• Gemini 3.1 Pro: 12.9%
• Claude Opus 4.8: 12.0%
• Seed 1.8 (open source): 39.7%

Models swing between blind trust and total skepticism rather than exercising calibrated judgment.

Hidden Cost of Existing Defenses

Four existing methods—prompt-defense, SFT, standard DPO, and OPSD—improve accuracy under misleading context, but all also degrade clean and correct-context accuracy. They teach "distrust everything" rather than "judge when to trust."

Analogy: a child taught never to cross the street has zero accident rate, but loses the ability to cross at all.

SCOPE: Signal-Counterfactual Preference Optimization

SCOPE balances DPO preference pairs across all four conditions—misleading teaches resistance; clean, correct-context, and irrelevant conditions teach preservation. Crucially, the four pairs per question are matched, so the model learns *condition-conditional* trust rather than blanket distrust.

Ablation: Matching Is the Load-Bearing Wall

• Unmatched or random pairs cause large drops in misleading accuracy, overall accuracy, and SC2W.
• Removing any one of the three control conditions weakens preservation in that condition.

This echo findings from the Regression Tax and Looping Is Not Reliability papers: paired structure reveals capability more honestly than pass rates.

Results

SCOPE significantly reduces SC2W on major open-source models while preserving accuracy across the other three conditions. The learned behavior transfers zero-shot to three external benchmarks, indicating real selective-trust acquisition rather than MIST overfitting.

Position in the Literature

The core lesson is structural:

> You optimize what you measure; what you don't measure is where problems hide.

This resonates with several recent works:

• Epanorthosis: LLMs reproduce ancient rhetorical patterns because RLHF rewards confident emphasis.
• Token Budget: CoT reasoning bimodality is encoded early in representations.
• QuantiBias: Quantization introduces 24–27% bias in standard safety-check blind spots.
• MIST: resistance training creates false robustness in the blind spot of context-trust.

The unifying law: single-dimensional training signals produce single-dimensional capabilities, while the real world demands multi-dimensional judgment. SCOPE's paired counterfactual design supplies such a signal, binding "trust" and "distrust" together.

Honest Limitations

• MIST is human-annotated with limited scale, despite dual-review arbitration.
• SCOPE training requires four-condition paired data, raising cost for resource-constrained teams.
• Experiments focus on open-source models; applicability to closed models via API distillation is open.

A Deeper Implication

The central question for alignment is not *how to make models obey*, but *how to make models judge when to obey*. RLHF can produce sycophancy; prompt-defense can produce blanket distrust. Both failures are isomorphic: univariate training yields univariate ability.

SCOPE's multi-dimensional paired training—simultaneously teaching *when to believe* and *when not to*—is a template likely applicable to broader alignment challenges beyond context trust.

Closing

MIST turns a vague dilemma into a measurable metric (SC2W) and an optimizable objective (paired counterfactual preference). Its core finding—that resistance training manufactures false robustness—is a warning every RLHF-era alignment method should heed. A model that learns to distrust everything is not stronger; it is broken. Real robustness is knowing when to trust.

---

Paper: https://arxiv.org/abs/2608.06377 HTML: https://arxiv.org/html/2608.06377v1 Code: https://github.com/worldbench/SCOPE