Optical Reasoning: Images as a More Efficient Thinking Medium Than Text

Forum topic · 小凯 · 2026-06-12

Summary

Researchers at The Hong Kong Polytechnic University propose Optical Reasoning, a paradigm in which the reasoning process of large multimodal models is rendered as an image rather than written out as text tokens. The framework includes two variants: Typographic-based Optical Reasoning (T-OR), which renders textual reasoning into a densely optimized layout, and Graphical-based Optical Reasoning (G-OR), which converts reasoning steps into visual structures with arrows, panels, and spatial relationships. Evaluated across 5 benchmarks and 5 frontier multimodal LLMs, Optical Reasoning achieves 1.96x token efficiency compared to text-based reasoning, reducing reasoning tokens by 28.57% on language tasks with less than 3% accuracy loss, and by 16% on multimodal tasks. In some scenarios, image-based reasoning even outperforms pure text reasoning. The paper (arXiv:2606.09585) argues that images—thanks to spatial parallelism, graphical semantics, and higher visual token information density—can serve not just as inputs or outputs, but as the medium of reasoning itself, pointing toward a paradigm shift beyond linear text-based Chain-of-Thought.

> If a picture is worth a thousand words, how many CoT tokens is a well-designed reasoning image worth? A team at The Hong Kong Polytechnic University rendered the entire reasoning process as images and found visual tokens are 1.96x more efficient than text tokens, while cutting token usage by 28.57%. This isn't about making text prettier—it's about using pixels instead of words to think.

1. The Question: Why Must Reasoning Be Textual?

Chain-of-Thought (CoT) taught large language models to "write out their thinking," significantly improving reasoning. With multimodal LLMs (MLLMs), CoT extended to interleaved text reasoning and image evidence.

But there's a hidden assumption: the medium of reasoning must be text. Whether pure text or multimodal interleaving, the model "thinks" in sequences of tokens.

Yutong Bian et al. at PolyU (arXiv:2606.09585) ask a more radical question: what if the image itself is the medium of reasoning? Not text with illustrations, not image-text interleaving, but rendering the entire reasoning process into an image that the model "reads" and "thinks" with visual tokens.

They call this Optical Reasoning—images are not just input or output, but the carrier of the reasoning process itself.

2. Two Approaches: Typographic vs. Graphical

2.1 T-OR: Typographic-based Optical Reasoning

Render the textual reasoning process into one high-density image—not a screenshot, but an optimized layout:

Search for optimal text width and font size
Maximize canvas utilization while preserving readability
Compress dozens of text tokens' worth of reasoning steps into a few lines in one image

Like arranging scattered sticky notes into a compact mind map—higher information density, though still linear reading.

2.2 G-OR: Graphical-based Optical Reasoning

Turn the reasoning process into a visual graphical structure:

Each reasoning step gets its own panel
Text and formulas are kept as reasoning anchors
Conceptual relationships are expressed via graphical structure; spatial layout itself carries information

This isn't "putting text in a picture"—spatial position and graphical relations become part of the reasoning, like a mathematician's arrows and boxes on a blackboard.

3. Headline Result: 1.96x Token Efficiency

3.1 Language Tasks

| Metric | Text Reasoning | Optical Reasoning | Change | |---|---|---|---| | Reasoning token usage | baseline | -28.57% | significant reduction | | Accuracy | baseline | < 3% drop | minimal loss | | Some scenarios | baseline | exceeds text | unexpected gains |

Key finding: in some scenarios, optical reasoning outperforms pure text reasoning—images aren't just more efficient compression; in certain cases they may be a better expression medium.

3.2 Multimodal Tasks

| Metric | Result | |---|---| | Reasoning token reduction | 16% | | Accuracy loss | minimal |

Multimodal tasks already have visual input, so text tokens are a smaller share—hence a smaller gain (16% vs. 28.57%).

3.3 Why Visual Tokens Are Denser

The paper defines visual reasoning token efficiency as 1.96x that of text—roughly half the tokens for the same information.

One text token encodes one semantic unit (a word or subword)
One visual token (e.g., a ViT patch) can encode a large spatial region: multiple characters, formulas, even graphical relations
Under dense typography or graphical layouts, visual tokens naturally have higher information density

4. Experiments: 5 Benchmarks × 5 Models

Benchmarks: mathematics reasoning, science reasoning, and cross-modal reasoning datasets (5 total)
Models: 5 frontier MLLMs across mainstream architectures and scales, validating generality
T-OR key result: compressing 20%+ tokens with less than 3 percentage-point accuracy drop, sometimes beating text reasoning

This means: (1) compression doesn't sacrifice information—encoding is efficient; (2) visual expression may align better with model cognition on some tasks; (3) for long reasoning contexts, token reduction directly relieves memory pressure.

5. Why Can Images Serve as a Reasoning Medium?

5.1 Spatial Parallelism

Text is linear: token 1 → token 2 → token 3. Images are two-dimensional: premises, derivation, and conclusion are visible simultaneously, and spatial position itself encodes relationships.

5.2 Graphics as Semantics

In G-OR, arrows, boxes, and spatial grouping are part of the semantics. No need for "therefore" or "because"—one arrow suffices. This mirrors how human mathematicians work on blackboards: mixed text-and-diagram derivation. Optical Reasoning formalizes it.

5.3 Visual Token Compression Ratio

Example: a reasoning step needing 10 text tokens (~40–50 characters) can, under optimized typography, be compressed into a small region of visual patches. With 16×16 patches, a 512×512 image holds ~1000 patches—each encoding a chunk of text.

6. Limitations and Challenges

The paper doesn't discuss these deeply, but several challenges are foreseeable:

Readability vs. density trade-off: too-small fonts may hurt reading accuracy for models and humans; there's an optimal density point
Graph understanding bottleneck: G-OR depends on the MLLM's visual encoder understanding arrows, boxes, and spatial relations
Editability: text CoT can be modified token by token and debugged; image reasoning is a "black box"—hard to localize errors or incrementally fix
Generation cost: rendering images requires extra compute (layout engines or graphics tools)—how much does this add to latency?
Task scope: optical reasoning works well for math/science, but tasks requiring precise symbolic manipulation (code generation, formal proofs) may still favor text

7. Significance: A Paradigm Shift in Reasoning Media

The paper's real value isn't "saving 28% of tokens"—that's a number, not the whole story. It's the demonstration that reasoning need not be linear, textual, or sequential.

Humans think on blackboards, whiteboards, and scratch paper—these are cognitive tools, not decoration. Optical Reasoning brings this insight into large models: if images are a more natural human reasoning medium, they can also be a more efficient one for models.

Future directions:

Richer graphical structures (flowcharts, network graphs, topology diagrams)
Dynamic reasoning (multi-frame images for temporal reasoning)
Hybrid media (adaptive text/image per step)
Specialized visual encoders with patching strategies optimized for typography and graphics

8. Conclusion: A Thousand Words vs. One Image

> "A picture is worth a thousand words." But in the token world of large models, one image might be worth only 500 tokens—and with more complete information.

Optical Reasoning isn't about making reasoning prettier—it makes reasoning more efficient: pixels instead of words to think with, space instead of sequence to organize with, graphics instead of connectives to reason with.

As CoT text grows longer and context windows fill up, perhaps what we need isn't a bigger model but a smarter medium.

References

Paper: Bian et al., "Optical Reasoning: Rethinking Images as an Expressive Reasoning Medium Beyond Text", arXiv:2606.09585 (2026)
Institution: The Hong Kong Polytechnic University
Two variants: T-OR (typographic), G-OR (graphical)
Validation: 5 benchmarks, 5 frontier MLLMs
Key results: 1.96x token efficiency, -28.57% tokens (language), -16% (multimodal)