Image Generators Are Generalist Vision Learners: How Vision Banana Turns AI Painting into Perception

Forum topic · 小凯 · 2026-05-22

Summary

This zhichai.net post analyzes the paper 'Image Generators are Generalist Vision Learners' by Kaiming He and colleagues at Google Research, which argues that image generators can serve as general-purpose vision learners. The model, Vision Banana, is built on Google DeepMind's Nano Banana Pro (NBP) diffusion engine and reframes perception tasks—depth estimation, semantic segmentation, surface normal estimation—as image generation problems. Depth is encoded into RGB via a non-linear log mapping, d_norm = C · log(1 + λ · D), preserving accurate physical scale after inversion. On benchmarks such as ADE20K segmentation (reported to outperform SAM 3), metric depth estimation (0.929 delta_1 vs Depth Anything V3's 0.918 on NYU, ETH3D, and KITTI), and surface normal estimation (vs Lotus-2), the generator reportedly beats specialist models. Notably, its 3D capabilities derive entirely from synthetic data, outperforming MoGe-2 by about 20% despite never seeing real physical depth. Using lightweight instruction tuning, the approach suggests generation and understanding share a common foundation, pointing toward unified world models for robotics and autonomous driving.

When Image Generators Become Generalist Vision Learners

*Translated and adapted from a zhichai.net forum post. Note: the post describes a paper dated 2026; figures below are as reported by the original author.*

AI image generators have long been praised for their ability to create something from nothing. But is photorealistic output all they can do? A paper titled "Image Generators are Generalist Vision Learners" — authored by Kaiming He and colleagues at Google Research — argues that the real ambition of image generators is not painting, but *seeing*.

Perception as Generation

The model in question, Vision Banana, may look like an image generator, but it functions as an all-purpose vision system. Instead of building separate heads for individual tasks (depth, segmentation, etc.), every vision problem is converted into an image generation task:

Want a depth map? It paints one.
Want a segmentation map? It paints one.

> What is Generalist Vision Learning? > A paradigm in which a model is not trained for one specific task (e.g., face recognition), but instead learns general features from large-scale data so it can tackle any visual problem.

The Math: Non-linear Depth Mapping

How do you paint the physical world's unbounded depth into RGB pixels? Vision Banana compresses true distance *D* into color channels using:

d_{norm} = C \cdot log(1 + \lambda \cdot D)

The logarithmic transform emphasizes near-field precision (mirroring biological vision) while still allowing exact physical dimensions to be recovered via the inverse function after generation.

Benchmark Results: Generalist Beats Specialists

| Task | Metric | Vision Banana (generalist) | Specialist baseline | |---|---|---|---| | Semantic segmentation (ADE20K) | mIoU (higher better) | Best reported | SAM 3 (outperformed) | | Metric depth estimation | δ₁ accuracy | 0.929 | Depth Anything V3 (0.918) | | Surface normal estimation | Mean angular error (lower better) | Best reported | Lotus-2 (outperformed) |

Even more striking: the model's 3D understanding is trained entirely on synthetic data. Having never observed real physical depth, it reportedly outperforms MoGe-2 by about 20% on real-world tests.

> The Synthetic Data Miracle > Training purely on simulated, computer-generated scenes while achieving strong generalization in the real world.

Toward a World Model

The results suggest generation and understanding are two sides of the same coin. To paint a convincing cup, a model must internally grasp the cup's 3D structure, materials, and context. Vision Banana hints at a future where AI is not just a word-guesser or doodler but a world model — a foundation that could underpin robotics and autonomous driving.

References & Key Facts (as cited in the post)

1. Paper: He, K., Xie, S., et al. (2026). *Image Generators are Generalist Vision Learners.* arXiv:2604.20329. 2. Base model: Google DeepMind's Nano Banana Pro (NBP) diffusion engine. 3. Method: Lightweight instruction-tuning that turns the generator into a zero-shot perceiver. 4. Benchmarks: Dominant results on NYU, ETH3D, and KITTI datasets. 5. Team: Collaboration between Kaiming He's lab and Google DeepMind.