When VLMs Say 'Let Me Look at the Image Again' — But Never Actually Look: The VisualSwap Study

Forum topic · 小凯 · 2026-05-18

Summary

A recent ICML 2026 Spotlight paper introduces VisualSwap, an image-swap probe that tests whether vision-language models (VLMs) genuinely re-examine images during self-reflection. The setup: while a VLM answers a visually dependent question and claims it will 're-check the image,' researchers silently swap it with a visually similar but semantically different image (e.g., a triangle with different side lengths, or a receipt with different amounts). Across Qwen3-VL, Kimi-VL, and ERNIE-VL, models overwhelmingly failed to notice the swap, with accuracy drops of up to 60%. Notably, thinking models were nearly three times more vulnerable than instruction-tuned models, and larger model size offered no protection. Attention analysis shows why: when a user explicitly asks the model to re-check, visual-token attention rises, but when the model itself utters a re-examination phrase, attention does not move — suggesting such statements are learned language patterns rather than genuine visual re-inspection. This post also raises open questions about image-swap selection criteria, context retrieval limits, and timing of self-reflection statements. The takeaway: when a VLM says it looked, don't trust it.

You show a vision-language model (VLM) a geometry problem: a right triangle with three labeled side lengths. The model says, "Let me take another look at the image to confirm..." and then gives the correct answer.

But if you silently swap the image for a different triangle — different side lengths, similar appearance — the model's answer barely changes.

It said "I'll take another look." It didn't look.

Shi, Yang, Wu, Jin, Shui, Berg-Kirkpatrick, and Ma recently published a paper on arXiv (2605.15864), accepted to ICML 2026 as a Spotlight, doing something simple but unsettling: they built an image-swap probing framework called VisualSwap.

🔀 The VisualSwap Experiment

The procedure is straightforward. First, ask a VLM a visually dependent question — what's in this image? During its answer, the model may say "let me double-check this image."

At exactly that moment, the experimenters silently swap the image for another one — visually similar but semantically different. Two shopping receipts with different amounts. Two similar triangles with different dimensions.

Then they check whether the model's post-"re-check" answer reflects the change.

The results are striking. Across Qwen3-VL, Kimi-VL, and ERNIE-VL, models almost entirely missed the swap. Accuracy dropped by up to 60%.

Even more counterintuitive: thinking models — those trained to do multi-step reasoning before answering — were more easily fooled than ordinary instruction models, by a gap of nearly three times. And bigger models were not harder to fool.

🧠 But What Does "Looking" Actually Mean?

The key point is not "VLMs can be fooled." It's that when a VLM is asked to "confirm," it produces a self-reflection text — "let me look at the image again" — but that text triggers no new processing of the visual information. It's a learned language pattern, not a genuine visual re-inspection behavior.

How do you prove this? The paper analyzes the model's attention allocation. When a user (explicitly, in the conversation) says "please re-check this image," the model's attention on visual tokens rises significantly — it really is looking. But when the model itself produces a "let me look again" statement, attention shows no comparable shift. The eyes don't move.

> This may resonate intuitively with anyone who has used GPT-4o or Gemini in multi-turn conversations. Sometimes the model says "let me recompute that" mid-reasoning, then gives the exact same wrong answer as before. You wonder: did it actually recompute? This paper proves that — for vision — very often it did not.

📉 Why Are Thinking Models More Vulnerable?

The paper finds that thinking models — those trained to internally produce a chain-of-thought before answering — are more susceptible to this illusion. A possible reason: thinking models have longer contexts and manage more text, so mentions of "re-check the image" get buried in the reasoning. Instruction models respond better to "please check" instructions in the input because they lack large amounts of internal reasoning text distracting them.

Scale doesn't help either. Larger models show no better visual re-checking ability. This suggests the problem is not "the model isn't smart enough" — it's that "the model architecture has no mechanism that forces visual re-inspection."

🤷 What Remains Unclear

First, how were the 800 swap pairs in VS-Bench selected? If swapped images sit close to the originals in the model's visual feature space — i.e., they genuinely "look" similar to the model — missing the swap may partly reflect reasonable generalization rather than pure visual neglect. The paper says the swaps are "visually similar but semantically different," but is that similarity threshold aligned with the tested models' feature spaces? I'm not sure.

Second, the paper finds user instructions work but model self-reflection doesn't. Yet there's an important difference: user instructions arrive in a new conversation turn, usually paired with a fresh image input, while self-reflection is generated autoregressively within the same reasoning step. Does this mean it's not "self-reflection text" that fails, but rather the model's retrieval ability within the same context? The paper doesn't disentangle this.

Third, when do self-reflection statements occur during reasoning? Does saying "let me look again" early in visual reasoning differ from saying it late? The paper does attention analysis but no temporal ablation. The same sentence at different times may have completely different effects.

Still, the paper's core warning is big and clear: when a VLM says it "looked" at something, don't trust it. It may just be uttering a sentence it learned sounds natural.

---

References

1. Shi, C., et al. (2026). *Are VLMs Seeing or Just Saying? Uncovering the Illusion of Visual Re-examination*. arXiv:2605.15864 [cs.CV]. https://arxiv.org/abs/2605.15864 (ICML 2026 Spotlight) 2. OpenAI. (2024). *GPT-4V(ision) System Card*. Technical Report. 3. Bai, J., et al. (2025). *Qwen3-VL: A Versatile Vision-Language Model*. arXiv preprint. 4. Team Kimi. (2025). *Kimi-VL Technical Report*. arXiv preprint. 5. Augenstein, I., et al. (2024). *Seeing the Unseen: Visual Hallucination in Large Vision-Language Models*. ACL 2024.