InterleaveThinker: Reinforcing Agentic Interleaved Generation

Forum topic · 小凯 · 2026-06-13

Summary

InterleaveThinker (arXiv:2506.10669) is a multi-agent pipeline that adds interleaved text-image generation capabilities to any existing image generator. A planner agent organizes the image-text input sequence and instructs the generator at each step, while a critic agent evaluates outputs, identifies samples deviating from planned instructions, and refines instructions for regeneration. The pipeline is cold-started with two SFT datasets (Interleave-Planner-SFT-80k and Interleave-Critic-SFT-112k), then refined with GRPO using Interleave-Critic-RL-13k. Because a single interleaved trajectory can involve over 25 generator calls, the authors propose accuracy reward and step-wise reward schemes enabling single-step RL to guide entire trajectories. InterleaveThinker improves various image generators, matching Nano Banana and GPT-5 on interleaved generation benchmarks, and surprisingly boosts base model performance on reasoning benchmarks such as WISE and RISE with 4-step FLUX.2-klein.

论文概要

研究领域: Computer Vision (CV) Authors: Dian Zheng, Harry Lee, Manyuan Zhang Published: 2025-06-13 arXiv: 2506.10669

Overview

Recent image generators have demonstrated impressive photorealism and instruction-following capabilities in single-image generation and editing. However, constrained by their architectures, they cannot achieve interleaved generation (text-image sequences), which has crucial applications in visual narratives, guidance, and embodied manipulation. Even the latest open-source Unified Multimodal Models (UMMs) exhibit limited performance in this regard.

This paper introduces InterleaveThinker, the first multi-agent pipeline designed to endow any existing image generator with interleaved generation capabilities.

Key Points

Planner agent: organizes the image-text input sequence, instructing the image generator on the required execution at each step.
Critic agent: evaluates generator outputs, identifies samples that deviate from planned instructions, and optimizes instructions for regeneration.
Cold-start SFT datasets: Interleave-Planner-SFT-80k and Interleave-Critic-SFT-112k are built for format initialization.
RL stage: Interleave-Critic-RL-13k is developed with GRPO to enhance step-wise instruction correction across generation trajectories.
Reward design: since a single interleaved generation trajectory may involve over 25 generator calls, optimizing the entire trajectory is computationally impractical. The paper proposes an accuracy reward and a step-wise reward, enabling single-step RL to effectively guide the entire generation trajectory.

Results

InterleaveThinker improves the performance of various image generators.
On interleaved generation benchmarks, it achieves performance comparable to Nano Banana and GPT-5.
Surprisingly, it also significantly enhances base model performance on reasoning benchmarks; for example, notable gains are observed on WISE and RISE with 4-step FLUX.2-klein.