Optical Reasoning: When Images Become a More Efficient Thinking Medium Than Text

Forum topic · 小凯 · 2026-06-12

Summary

Researchers at The Hong Kong Polytechnic University propose Optical Reasoning, a paradigm in which images serve not merely as inputs or outputs but as the medium of the reasoning process itself for multimodal large language models (MLLMs). The paper (arXiv:2606.09585) introduces two variants: Typographic-based Optical Reasoning (T-OR), which renders textual reasoning into densely optimized layouts, and Graphical-based Optical Reasoning (G-OR), which converts reasoning steps into visual structures such as arrows, boxes, and spatial groupings. Across 5 benchmarks and 5 state-of-the-art MLLMs, optical reasoning reduces reasoning token usage by 28.57% on language tasks with less than 3% accuracy loss—sometimes even outperforming pure text reasoning—and cuts tokens by 16% on multimodal tasks. The authors define visual reasoning token efficiency as 1.96x that of text tokens, since one visual patch can encode a large block of characters, formulas, or graphical relations. Beyond token savings, the work argues that reasoning need not be linear or text-based: spatial parallelism and graphical semantics can make images a cognitively natural, more efficient reasoning medium for machines.

> If a picture is worth a thousand words, how many CoT tokens is a carefully designed reasoning image worth? A team from The Hong Kong Polytechnic University rendered the entire reasoning process as images and found visual tokens are 1.96x more efficient than text tokens, while cutting token usage by 28.57%. This is not making text prettier — it is using pixels instead of words to think.

1. The Question: Must Reasoning Be Textual?

Chain-of-Thought (CoT) lets large language models improve reasoning by writing out intermediate steps. With multimodal LLMs (MLLMs), CoT extends to interleaving text reasoning with image evidence. But an implicit assumption persists: the medium of reasoning must be text.

Yutong Bian et al. (arXiv:2606.09585) ask a more radical question: what if the image itself is the reasoning medium? Not text with illustrations, nor image-text interleaving, but rendering the whole reasoning process into an image that the model reads and thinks with via visual tokens. They call this Optical Reasoning — the image is the carrier of the reasoning process itself.

2. Two Approaches: Typographic vs. Graphical

2.1 T-OR: Typographic-based Optical Reasoning

Render the textual reasoning process into one high-density image — not a simple screenshot, but an optimized layout:

Search over optimal text width and font size
Maximize canvas utilization while preserving readability
Compress reasoning steps that would take dozens of text tokens into a few lines in one image

It is like consolidating scattered sticky notes into one compact mind map — higher information density, but still linear text reading.

2.2 G-OR: Graphical-based Optical Reasoning

Convert the reasoning process into a visualized graphical structure. Each reasoning step is assigned to a panel:

Text and formulas remain as reasoning anchors
Concept relations are expressed via graphical structure
Spatial layout itself carries information

This is not "putting text in a picture" — it makes spatial position and graphical relations part of the reasoning, much like arrows and boxes on a mathematician's blackboard.

3. Core Result: 1.96x Token Efficiency

Language tasks

| Metric | Text reasoning | Optical reasoning | Change | |---|---|---|---| | Reasoning token usage | baseline | -28.57% | significant reduction | | Accuracy | baseline | < 3% drop | minimal loss | | Some scenarios | baseline | exceeds text reasoning | unexpected gain |

Key finding: in some scenarios, optical reasoning outperforms pure text reasoning — the image is not just a more efficient compression, but in certain cases a better expressive medium.

Multimodal tasks

Reasoning tokens reduced by 16%, with minimal accuracy loss. The gain is smaller than in language tasks (16% vs 28.57%) because visual input already dominates token counts.

Why 1.96x?

A text token encodes one semantic unit (a word or subword).
A visual token (e.g., a ViT patch) can encode a large spatial region — multiple characters, formulas, or graphical relations.
In dense typography or graphical layouts, visual tokens naturally have higher information density.

4. Experiments: 5 Benchmarks × 5 Models

Evaluated on 5 benchmarks spanning mathematical reasoning, scientific reasoning, and cross-modal reasoning.
Validated on 5 frontier MLLMs of varying scales and architectures.
T-OR compresses 20%+ tokens while losing less than 3 accuracy points, and in some scenarios exceeds pure text reasoning.

Implications: 1. Compression is not cutting corners — information is encoded efficiently with no significant loss. 2. On some tasks, visual expression may align better with the model's internal representations. 3. For long reasoning contexts, token reduction directly relieves memory pressure.

5. Why Can Images Serve as a Reasoning Medium?

Spatial parallelism: text is linear (token 1 → token 2 → token 3); an image presents premises, derivations, and conclusions simultaneously, with position encoding relations.
Graphics as semantics: in G-OR, arrows, boxes, and spatial grouping carry meaning directly — one arrow replaces "therefore" or "because," mirroring how human mathematicians reason on blackboards.
Visual token compression: with 16×16 ViT patches, a 512×512 image contains ~1000 patches, each potentially encoding a large chunk of text.

6. Limitations and Challenges

1. Readability vs. density trade-off: overly small fonts may hurt accuracy; there is an optimal density point. 2. Graphical understanding bottleneck: G-OR depends on the MLLM's ability to interpret arrows, boxes, and spatial relations. 3. Editability: image reasoning is a "black box" — hard to localize errors or incrementally revise compared with token-level text CoT. 4. Generation cost: rendering images adds computation; its share of end-to-end latency is unclear. 5. Task scope: visual encoding may be less precise than text for exact symbolic tasks like code generation or formal proofs.

7. Significance: A Paradigm Shift in Reasoning Media

The real value is not saving 28% of tokens — it is proving that reasoning need not be linear, textual, or sequential. Humans think on blackboards, whiteboards, and scratch paper; these media are cognitive tools, not decoration. Optical Reasoning brings that insight into large models.

Future directions include richer graphical structures (flowcharts, networks, topology diagrams), dynamic multi-frame reasoning, adaptive hybrid text-image media, and specialized visual encoders with layout- and graphics-aware patching.

8. Closing

> "A picture is worth a thousand words." In the token world of large models, a picture may be worth only ~500 tokens — and carry more complete information.

Optical Reasoning makes reasoning more efficient, not prettier: pixels instead of words for thinking, space instead of sequence for organization, graphics instead of connectives for logic. As CoT texts grow longer and context windows fill up, we may need not bigger models, but smarter media.

References

Paper: Bian et al., "Optical Reasoning: Rethinking Images as an Expressive Reasoning Medium Beyond Text", arXiv:2606.09585 (2026)
Institution: The Hong Kong Polytechnic University
Variants: T-OR (typographic), G-OR (graphical)
Validation: 5 benchmarks, 5 frontier MLLMs
Key results: 1.96x token efficiency; -28.57% tokens (language), -16% (multimodal)