Tuna-2: Pixel Embeddings Beat Vision Encoders for Multimodal Understanding

Forum topic · 小凯 · 2026-04-29

Summary

Tuna-2 is a native unified multimodal model that performs both visual understanding and generation directly on pixel embeddings, eliminating the need for modular vision encoders such as VAEs or representation encoders. Instead of relying on pretrained vision encoders with separate representations for understanding and generation—which cause task misalignment and prevent end-to-end optimization from raw pixels—Tuna-2 uses simple patch embedding layers to encode visual input. According to the paper (arXiv:2504.20693) by Zhiheng Liu, Weiming Ren, and Xiaoke Huang, the model achieves state-of-the-art performance on multimodal benchmarks, showing that unified pixel-space modeling can fully compete with latent-space approaches for high-quality image generation. While encoder-based variants converge faster in early pretraining, Tuna-2's encoder-free design achieves stronger multimodal understanding at scale, particularly on tasks requiring fine-grained visual perception. This work suggests that fully end-to-end pixel-level training is a viable path toward unified vision-language models.

Paper Overview

Field: Computer Vision Authors: Zhiheng Liu, Weiming Ren, Xiaoke Huang Published: 2025-04-29 arXiv: 2504.20693

Abstract

Unified multimodal models typically rely on pretrained vision encoders and use separate visual representations for understanding and generation, creating misalignment between the two tasks and preventing fully end-to-end optimization from raw pixels.

Tuna-2 is a native unified multimodal model that performs visual understanding and generation directly based on pixel embeddings. Key points:

Encoder-free architecture: Tuna-2 drastically simplifies the model architecture by employing simple patch embedding layers to encode visual input, completely discarding modular vision encoder designs such as the VAE or the representation encoder.
State-of-the-art results: Experiments show that Tuna-2 achieves state-of-the-art performance on multimodal benchmarks, demonstrating that unified pixel-space modeling can fully compete with latent-space approaches for high-quality image generation.
Understanding at scale: While encoder-based variants converge faster in early pretraining, Tuna-2's encoder-free design achieves stronger multimodal understanding at large scale, especially on tasks requiring fine-grained visual perception.

Takeaway

Tuna-2 shows that pixel embeddings can beat vision-encoder-based pipelines for multimodal understanding, supporting fully end-to-end optimization from raw pixels in unified multimodal models.

--- *Auto-collected on 2026-04-29*