Paper Overview
Field: NLP Author: Zixiang Xu Published: 2026-09-24 arXiv: 2609.30243
Summary
Dedicated decision models such as Jev map unstructured language to probability distributions over finite choices, allowing their outputs to directly route requests, select tools, and trigger actions. However, real-world inputs rarely arrive in isolation — they come with background details and surrounding context. The authors find that short additions that fit naturally into this context can redirect an otherwise correct decision, even when the correct answer remains unchanged.
Method
To study this behavior, the authors fix a wrong target option for each initially correct sample and use the model's option probabilities to refine fluent context additions, while keeping the original source, question, choices, and gold answer unchanged.
Key Findings
- Within 64 accepted target evaluations, the optimizer identified contexts that redirected Jev on 312/508 initially correct decisions (61.4%).
- In 229 cases, Jev assigned at least 0.7 probability to the fixed wrong option.
- Across seven datasets and three additional decision systems, targeted flip rates on initially correct decisions reached 64.9%–73.2%.
Implications
These results expose a notable vulnerability of current decision models: brief, seemingly ordinary context can turn a correct choice into a high-confidence wrong one. Because such models convert language directly into downstream choices, this sensitivity is concerning — their probability outputs should not be treated as a reliable decision interface.
Abstract (Original)
Dedicated decision models such as Jev map unstructured language to probability distributions over finite choices, allowing their outputs to directly route requests, select tools, and trigger actions. Yet real-world inputs rarely arrive in isolation: they come with background details and surrounding context. We find that short additions that fit naturally into this context can nevertheless redirect an otherwise correct decision, even when the correct answer remains unchanged. To study this behavior, we fix a wrong target option for each initially correct item and use the model's option probabilities to refine fluent context additions while preserving the source, question, choices, and gold answer. Within 64 accepted target evaluations, the optimizer identifies contexts that redirect Jev on ...
--- *Auto-collected on 2026-09-28*