LLaDA2.0-Uni: A Unified Discrete Diffusion LLM for Multimodal Understanding and Generation

Forum topic · 小凯 · 2026-04-24

Summary

LLaDA2.0-Uni is a unified discrete diffusion large language model (dLLM) from Inclusion AI that supports multimodal understanding and generation in a single natively integrated framework. Its architecture combines a fully semantic discrete tokenizer, a MoE-based dLLM backbone, and a diffusion decoder. Continuous visual inputs are discretized via SigLIP-VQ, enabling block-level masked diffusion over both text and vision inputs in the backbone, while the diffusion decoder reconstructs visual tokens into high-fidelity images. Inference efficiency is improved beyond parallel decoding through prefix-aware optimizations in the backbone and few-step distillation in the decoder. Trained on carefully curated large-scale data with a tailored multi-stage pipeline, LLaDA2.0-Uni matches specialized vision-language models on multimodal understanding while delivering strong image generation and editing performance. Its native support for interleaved generation and reasoning establishes a scalable paradigm for next-generation unified foundation models. Paper: arXiv 2604.20796; code and models: https://github.com/inclusionAI/LLaDA2.0-Uni.

Paper Overview

Field: Computer Vision (CV)
Authors: Inclusion AI, Tiwei Bie, Haoxing Chen
Published: 2026-04-22
arXiv: 2604.20796

Abstract (English)

We present LLaDA2.0-Uni, a unified discrete diffusion large language model (dLLM) that supports multimodal understanding and generation within a natively integrated framework. Its architecture combines a fully semantic discrete tokenizer, a MoE-based dLLM backbone, and a diffusion decoder. By discretizing continuous visual inputs via SigLIP-VQ, the model enables block-level masked diffusion for both text and vision inputs within the backbone, while the decoder reconstructs visual tokens into high-fidelity images. Inference efficiency is enhanced beyond parallel decoding through prefix-aware optimizations in the backbone and few-step distillation in the decoder. Supported by carefully curated large-scale data and a tailored multi-stage training pipeline, LLaDA2.0-Uni matches specialized VLMs on multimodal understanding while delivering strong performance in image generation and editing. Its native support for interleaved generation and reasoning establishes a promising and scalable paradigm for next-generation unified foundation models.

Key Components

Fully semantic discrete tokenizer: SigLIP-VQ discretizes continuous visual inputs into tokens compatible with the language backbone.
MoE-based dLLM backbone: performs block-level masked diffusion over both text and vision inputs in a unified sequence.
Diffusion decoder: reconstructs visual tokens into high-fidelity images.

Efficiency

Prefix-aware optimizations in the backbone improve inference beyond naive parallel decoding.
Few-step distillation in the decoder reduces image generation cost.

Results & Availability

Matches specialized vision-language models on multimodal understanding benchmarks.
Strong performance on image generation and editing; supports native interleaved generation and reasoning.
Code and models: https://github.com/inclusionAI/LLaDA2.0-Uni

*Auto-collected on 2026-04-24.*