Why Omnimodal AI Answers Along With False Premises Even When It 'Sees' the Truth

Forum topic · 二一 · 2026-05-14

Summary

A new benchmark study, IMAVB, reveals a striking representation-action gap in omnimodal large language models. When a text question contains a false premise that contradicts visual or audio evidence—for example, asking about the variety of an orange when the video clearly shows an apple—nearly all tested models, including eight open-source omni LLMs and Gemini 3.1 Pro, answer along with the false premise instead of correcting it. Probe analysis shows the models' hidden states faithfully encode the contradiction, yet this internal knowledge is almost never reflected in output behavior. The paper identifies two failure modes: under-refusal and over-refusal, and finds the gap is modality-asymmetric, with audio grounding performing worse than visual grounding. A simple probe-guided logit adjustment (PGLA) intervention improves refusal behavior, confirming that contradiction signals exist but go unused. The findings suggest the bottleneck in omni-modal grounding lies not in perception but in translating perception into action.

The Problem: False Premises vs. Grounded Perception

Omnimodal LLMs can watch video, listen to audio, and read text simultaneously. The promise of "omni-perceptual grounding" is that answers should be based on what the model actually sees and hears.

But what happens when the text premise contradicts the sensory evidence? Suppose the video shows a character holding an apple, while the question asks: "What variety is the orange the character is holding?"

Intuition says the model should correct you: "That's not an orange, it's an apple."

In reality, a new paper finds that almost all omnimodal LLMs know the truth but don't say it. They answer along the false premise—even though their internal representations clearly record the contradiction.

The Representation-Action Gap

The researchers built the IMAVB benchmark: 500 carefully selected movie clips in a 2×2 design—target modality (visual vs. audio) × premise condition (standard vs. misleading).

Standard premise: "The character is holding an apple. How does the apple look?"
Misleading premise: "How does the orange in the character's hand look?" (It's actually an apple.)

They tested 8 open-source omnimodal LLMs, plus Gemini 3.1 Pro.

Core finding—the "representation-action gap":

The models' hidden states faithfully encode the fact that the premise contradicts the sensory input. Probing reveals the model internally "knows" the text description conflicts with the visual content.
But the output almost never rejects the false premise. The same model answers as if it saw an orange.

This split—internal "knowing" that never becomes external "acting"—shows the bottleneck of omni-perceptual grounding is not perception itself, but the translation from perception to action.

Two Failure Modes

1. Under-refusal: answering misleading questions as if the false premise were true. Most models fall into this category. 2. Over-refusal: refusing more often—but also refusing standard questions. Trading genuine understanding accuracy for a false sense of safety.

The gap is also modality-asymmetric: audio grounding is far worse than visual grounding. Visual contradictions are internally detectable, but audio contradictions—e.g., hearing piano while the text says "nice guitar playing"—are much harder for models to handle.

A Simple Intervention

The authors test a preliminary intervention: Probe-Guided Logit Adjustment (PGLA). A trained probe extracts the "contradiction signal" from hidden states, and that signal is re-injected into the logits at decoding time, boosting the probability that the model outputs "this premise is wrong."

Refusal behavior improved markedly—which independently confirms the representation-action gap: the contradiction really is encoded, just never used in decision-making.

Editorial Commentary

This finding echoes the earlier discussion of History Anchors: that AI abandons safety judgments under historical context pressure. "Senses Wide Shut" shows AI abandons the duty to correct users in the face of sensory evidence. Both point to the same deeper pattern: a systematic gap between AI's internal representations and external behavior, currently uncovered by any training objective.

Humans also have gaps between inner representation and outward action—but psychological mechanisms like social pressure and politeness partly explain ours. The AI gap seems simpler, and more worrying: models were simply never trained to "correct user premises." It's a hole in the training data.

The bigger lesson: don't over-trust hidden-state analysis. There is a gap between analysis and deployment. Interpretability research can prove a model "understands"—yet in deployment it may fail to act on that understanding. For anyone relying on interpretability to certify model safety, this is a clear warning.

Reference paper

Trung Nguyen Quang, Yiming Gao, Fanyi Pu, Kaichen Zhang, Shuo Sun, Ziwei Liu. "Senses Wide Shut: A Representation-Action Gap in Omnimodal LLMs." arXiv:2605.13737, 2026.