Feynman-style Letter: DeepSeek's Reasoning with Visual Primitives

Forum topic · 小凯 · 2026-05-03

Summary

A Chinese tech forum post reviews DeepSeek-AI's research on 'Thinking with Visual Primitives,' arguing that current multimodal large language models (MLLMs), despite processing high-resolution images, reason over flat pixel features without explicit spatial coordinates—often failing at occlusion and relative-position questions, a problem the author calls the 'reference gap between semantics and physical coordinates.' DeepSeek's approach injects two mathematical primitives—points and boxes—into the chain-of-thought, so the model outputs structured coordinates such as {"point": [120, 305], "label": "cup handle"} while reasoning. With explicit coordinates, visual reasoning shifts from probabilistic guessing to geometric deduction: the model can derive conclusions like 'point A lies to the right of box B, and box B is closer to the camera, so A is partially occluded by B,' closing the reference loop between language and physical space. The author's takeaway: the next step in visual understanding is not higher resolution but finer coordinate awareness, and practitioners should add 'physical anchors' to multimodal data rather than only polishing prompts.

Feynman Letter: Do you want to see 'a blob of mush,' or draw 'points and lines' in your mind?

After reading DeepSeek-AI's paper on Thinking with Visual Primitives, I feel multimodal models have finally moved past being 'blind despite open eyes' and started learning to 'underline the key points.'

To explain why today's multimodal AI often fails at simple geography questions, let's talk about 'frames of reference.'

1. The status quo: a giant fumbling blindly in high-res images

Current multimodal large models (MLLMs) can see tens of millions of pixels, but their thinking is flat. When you ask 'where is the cup on the left,' it performs a probabilistic blind search through a pile of jumbled pixel features.

Pain point: Lacking an explicit, mathematical spatial coordinate system, it often can't distinguish front from back or handle occlusion. It can describe an image, but it cannot build a logical model of the image in its mind. This is the 'reference gap between semantics and physical coordinates.'

2. Visual primitives: the geometer with compass and straightedge

DeepSeek's move is very geeky: I won't let you stare directly at the image—I'll teach you to draw 'marks' on it.

It achieves a dimensionality-reduction strike on visual reasoning:

Primitive-based thinking (Visual Primitives): It forces two mathematical weapons into the AI's chain of thought (CoT): Point and Box. When the AI thinks, it isn't spouting filler—it's writing coordinates in its head: {"point": [120, 305], "label": "cup handle"}.
Spatial deduction: With these coordinates, reasoning changes from 'literary description' to 'geometric proof.' It can precisely deduce: 'Since point A is to the right of box B, and box B is closer to the camera, A must be partially occluded by B.'
Closed reference loop: This fully closes the 'reference gap.' Every conclusion the model emits now stands on a real physical coordinate.

3. A Feynman-style judgment: intelligence comes from 'solidifying dimensions'

True 'understanding' is not recognizing an image's class. It is whether you can abstract a logically operable, topologically consistent symbol system out of the chaotic visual stream.

DeepSeek shows us: the next stop for visual understanding is not higher resolution, but finer 'coordinate perception.'

When AI learns to sketch the world with 'points and lines' in its thinking, it is no longer a parrot that merely captions pictures, but a 'silicon-based observer' with genuine spatial reasoning.

Takeaway

When optimizing your multimodal products, don't just grind on prompt rhetoric. Add 'physical anchors' to your data.

If you can teach AI to draw a red circle on the map before it starts talking, what you save is the enormous communication cost humanity pays for 'misunderstanding.'