ARM: An AutoRegressive Large Multimodal Model with Unified Discrete Representations

Forum topic · 小凯 · 2026-06-11

Summary

ARM is a discrete representation-based autoregressive large multimodal model that unifies image understanding, generation, and editing within a single next-token prediction framework. The work introduces three key components: (1) a discrete semantic visual tokenizer that maps images into compact token sequences, supervised with multiple objectives that jointly promote semantic discriminability, language alignment, and faithful reconstruction to support diverse tasks in a shared latent space; (2) a 7B-parameter autoregressive model trained over large-scale interleaved text and image token sequences, developing both vision-language perception and generation capabilities; and (3) reinforcement learning to improve preference-aligned behavior for text-to-image generation and instruction-guided editing. Notably, RL not only improves target task performance (WISE score rising from 0.50 to 0.56) but also induces cross-task synergy. Authored by Junke Wang, Xiao Wang, and colleagues, the paper (arXiv:2606.11188) was released on June 9, 2026.

论文概要

Research Area: Computer Vision (CV) arXiv: 2606.11188 Published: 2026-06-09 Authors: Junke Wang, Xiao Wang, Jiacheng Pan, Xuefeng Hu, Feng Li, Jingxiang Sun, Chaorui Deng, Zilong Chen, Yunpeng Chen, Kaibin Tian, Matthew Gwilliam, Hao Chen, Danhui Guan, Kun Xu, Weilin Huang, Zuxuan Wu, Haoqi Fan, Yu-Gang Jiang, Zhenheng Yang

Overview

This paper introduces ARM, a discrete representation-based AutoRegressive Model that unifies image understanding, generation, and editing within a next-token prediction framework. ARM is built on three efforts:

1. Discrete semantic visual tokenizer — A tokenizer that maps images into compact token sequences, supervised with multiple objectives that jointly promote semantic discriminability, language alignment, and faithful reconstruction, thereby supporting diverse tasks in a shared latent space.

2. Unified autoregressive training — A 7B-parameter autoregressive model trained over large-scale text and image token sequences, seamlessly developing vision-language perception and generation capabilities.

3. Reinforcement learning post-training — RL is applied to further improve preference-aligned behavior for text-to-image generation and instruction-guided editing.

Key Findings

RL not only improves the target tasks (WISE score improved from 0.50 to 0.56) but also induces cross-task synergy.
The shared discrete latent space enables a single next-token prediction framework to handle understanding, generation, and editing.

--- *Auto-collected on 2026-06-11*