G2U: How Multimodal Models Use Image Generation to Improve Image Understanding

Forum topic · 小凯 · 2026-05-18

Summary

A new framework called G2U (Generation-to-Understanding), presented by Tong, Chang, Yin, Liu, Fang, and Ma at CVPR 2026 Findings (arXiv:2605.15792), reverses the usual one-way relationship between visual understanding and generation in multimodal AI. Instead of only using understanding to guide generation, G2U makes visual generation an explicit intermediate step in the reasoning chain: the model first performs a generative operation on an input image—enhancing details, expanding context, or visualizing structure—then feeds the generated image back as new input and reasons over this self-generated visual thinking. The approach requires no retraining and no external tools, and across 12 benchmarks it consistently improves multimodal understanding. The authors also honestly note a limitation: self-generated visual thinking lacks stable task alignment, meaning what the model chooses to draw often does not match what the task actually requires. The forum author raises open questions about whether gains come from new visual information or merely ensemble effects of two-pass reasoning, and notes the substantial computational cost of generating before understanding.

The long-standing goal of multimodal AI has been a two-way synergy between visual understanding and visual generation: you understand images and can draw new ones, and drawing helps you understand better—or the reverse.

So far, however, this relationship has been one-directional. Understanding always guides generation—"I want a picture of a cat"—but generation rarely feeds back into understanding. "Draw a picture to help me better understand this picture" has been almost never done.

The G2U Framework

Tong, Chang, Yin, Liu, Fang, and Ma, in a CVPR 2026 Findings paper (arXiv:2605.15792), propose a framework called G2U (Generation-to-Understanding), making visual generation an explicit intermediate step in the understanding reasoning chain.

The idea is simple: the model first performs a generative operation on the input image—enhancing details, expanding context, or visualizing structure—then feeds the generated image back into the model as new input, and makes its understanding judgment based on this "self-generated visual thinking." No retraining and no external tools are required.

Results

Across 12 benchmarks, this reverse information flow consistently improves multimodal understanding.
Models effectively help themselves understand ambiguous images by "drawing a clearer version."

Limitations

The paper honestly reports a limitation: although the model can generate plausible edited images, its "self-generated visual thinking" lacks stable task alignment—what the model wants to draw to aid understanding often does not match what the task actually requires.

Open Questions from the Author

To what extent does "imagination" actually help the task, and to what extent does it just add visual redundancy?
Generate-then-understand improves accuracy—but is that because new visual information is added, or because of an ensemble effect from two rounds of inference? This is not fully disentangled.
Computational cost is substantial: generating an image and then understanding it is much slower than understanding directly.

References

1. Tong, Y., et al. (2026). *Reversing the Flow: Generation-to-Understanding Synergy in Large Multimodal Models*. arXiv:2605.15792 [cs.CV]. (CVPR 2026 Findings) 2. Team BAGEL. (2025). *BAGEL: Bootstrapping Unified Multimodal Generation and Understanding*. 3. Li, J., et al. (2023). *BLIP-3: Bootstrapping Language-Image Pre-training with Unified Understanding and Generation*. 4. OpenAI. (2024). *GPT-4V(ision)*. 5. Sun, Q., et al. (2024). *Generative Visual Reasoning with Multimodal Large Language Models*.