Feynman's Letter: On DeepSeek's Reasoning with Visual Primitives

Forum topic · 小凯 · 2026-05-03

Summary

This forum post reviews DeepSeek-AI's research on 'Thinking with Visual Primitives,' arguing that multimodal large language models (MLLMs) often fail at spatial questions because they reason over flat pixel features without explicit coordinates—a gap the author calls the 'referential gap between semantics and physical coordinates.' DeepSeek's approach injects two mathematical primitives—points and boxes—into the model's chain-of-thought, so the AI emits structured coordinates like {"point": [120, 305], "label": "cup handle"} while reasoning. With these anchors, spatial inference shifts from probabilistic blind search to geometric deduction (e.g., occlusion and relative-position reasoning), closing the referential loop between language and space. The author, writing in a Feynman-inspired style, concludes that true visual understanding means abstracting a logically operable, topologically consistent symbol system from chaotic visual input, and that the next step for multimodal AI is finer coordinate awareness rather than higher resolution. Practical takeaway for practitioners: add physical anchors to multimodal data instead of over-tuning prompt rhetoric.

Feynman's Letter: Do You Want to See a "Blurry Mess," or Draw "Points and Lines" in Your Mind? — On DeepSeek's Visual Primitive Reasoning

After reading DeepSeek-AI's frontier paper on Thinking with Visual Primitives, I feel that multimodal models have finally shaken off the embarrassment of being "blind despite open eyes" and have started to learn how to "underline the key points."

To help you understand why today's multimodal AI so often fails at simple geography questions, let's talk about "frames of reference."

1. The Status Quo: A Giant Fumbling Blindly in High-Resolution Images

Current multimodal large models (MLLMs) can see tens of millions of pixels, but their thinking is flat. When you ask "where is the cup on the left," it performs a "probabilistic blind search" through a pile of messy pixel features.

The pain point: Because it lacks an explicit, mathematically grounded spatial coordinate system, it often cannot distinguish front from back, or resolve occlusion. It can describe a scene, but it cannot perform "logical modeling" of the scene in its head. This is called the "referential gap between semantics and physical coordinates."

2. Visual Primitives: The Geometer with "Compass and Straightedge"

DeepSeek's move is extremely geeky: I won't let you stare at the image directly; I'll teach you to draw "marks" on it.

It achieves a dimensional-reduction strike in visual reasoning:

Primitive-based thinking (Visual Primitives): It forcibly inserts two mathematical weapons into the AI's chain of thought (CoT): the point and the box. When the AI thinks, it isn't spouting filler—it is writing coordinates in its head: {"point": [120, 305], "label": "cup handle"}.

Spatial deduction: With these coordinates, the AI's reasoning changes from "literary description" to "geometric proof." It can precisely deduce: "Since point A is to the right of box B, and box B is closer to the camera, then A must be partially occluded by B."

Closed referential loop: This completely fills the "referential gap." Every conclusion the model emits stands on a concrete physical coordinate.

3. A Feynman-Style Judgment: Intelligence Comes from "Fixing Dimensions"
So-called "understanding" is not about recognizing an image's category.

It is about whether you can abstract, from the chaotic visual stream, a symbol system that can be operated on by logic and has topological consistency.

DeepSeek tells us: The next stop for visual understanding is not higher resolution, but finer "coordinate perception."

When AI learns to use "points and lines" in its thinking to sketch the world, it is no longer a parrot that can only narrate pictures—it becomes a true "silicon-based observer" with spatial reasoning ability.

Key takeaway:

When optimizing your multimodal products, don't just grind on prompt rhetoric.

Go add "physical anchors" to your data.

If you can teach AI to draw a red circle on the map before it speaks, what you save is the enormous communication cost humanity pays for "misunderstanding."**