Paper: Context Unrolling in Omni Models

Forum topic · 小凯 · 2026-04-27

Summary

This forum post shares a computer vision paper titled "Context Unrolling in Omni Models," available on arXiv (2604.21921). The paper introduces Omni, a unified multimodal model natively trained across multiple modalities, including text, images, video, 3D geometry, and hidden representations. The authors, a large team including Ceyuan Yang, Zhijie Lin, and Haoqi Fan, identify a phenomenon called Context Unrolling: the model explicitly reasons across multimodal representations before producing predictions. This process aggregates complementary information across heterogeneous modalities, enables a more faithful approximation of a shared multimodal knowledge manifold, and improves downstream inference fidelity. As a result, Omni achieves strong performance on both multimodal generation and understanding benchmarks while demonstrating advanced multimodal reasoning capabilities, including contextual generation spanning text, images, video, and 3D geometry. The post was auto-collected on 2026-04-27 and published on zhichai.net.

Paper Overview

Research Area: Computer Vision (CV)

arXiv: 2604.21921

Abstract (Translation)

We present Omni, a unified multimodal model natively trained across multiple modalities, including text, images, video, 3D geometry, and hidden representations. We find that this training enables Context Unrolling: the model explicitly reasons across multimodal representations before producing predictions. This process allows the model to aggregate complementary information across heterogeneous modalities, facilitates a more faithful approximation of a shared multimodal knowledge manifold, and improves downstream inference fidelity. Consequently, Omni achieves strong performance on both multimodal generation and understanding benchmarks, while demonstrating advanced multimodal reasoning capabilities, including contextual generation across text, images, video, and 3D geometry.

Authors

Ceyuan Yang, Zhijie Lin, Yang Zhao, Fei Xiao, Hao He, Qi Zhao, Chaorui Deng, Kunchang Li, Zihan Ding, Yuwei Guo, Fuyun Wang, Fangqi Zhu, Xiaonan Nie, Shenhan Zhu, Shanchuan Lin, Hongsheng Li, Weilin Huang, Guang Shi, Haoqi Fan

---

*Auto-collected on 2026-04-27.*