InterleaveThinker: Reinforcing Agentic Interleaved Generation

Forum topic · 小凯 · 2026-06-14

Summary

InterleaveThinker (arXiv 2506.10669) is a multi-agent pipeline that enables any existing image generator to perform interleaved text-image sequence generation. A planner agent organizes the input sequence and instructs the generator at each step, while a critic agent evaluates outputs, detects deviations from planned instructions, and refines prompts for regeneration. The pipeline is cold-started with Interleave-Planner-SFT-80k and Interleave-Critic-SFT-112k datasets, then refined via GRPO on Interleave-Critic-RL-13k using accuracy rewards and stepwise rewards, making single-step RL feasible for trajectories with 25+ generator calls. On interleaved generation benchmarks, InterleaveThinker matches Nano Banana and GPT-5, and unexpectedly improves base model reasoning performance, with notable gains on WISE and RISE for 4-step FLUX.2-klein.

Paper Overview

Field: Computer Vision (CV)
Authors: Dian Zheng, Harry Lee, Manyuan Zhang
Released: 2025-06-13
arXiv: 2506.10669

Introduction

Recent image generators have demonstrated impressive photorealism and instruction-following capabilities in single-image generation and editing. However, constrained by their architectures, they cannot achieve interleaved generation (text-image sequences), which has crucial applications in visual narratives, guidance, and embodied manipulation. Even the latest open-source Unified Multimodal Models (UMMs) exhibit limited performance in this regard.

Method

InterleaveThinker is the first multi-agent pipeline designed to endow any existing image generator with interleaved generation capabilities:

Planner agent: organizes the image-text input sequence, instructing the image generator on the required execution at each step.
Critic agent: evaluates generator outputs, identifies samples that deviate from planned instructions, and refines instructions for regeneration.

Training uses format cold-start with two curated datasets:

Interleave-Planner-SFT-80k
Interleave-Critic-SFT-112k

Then GRPO is applied to develop Interleave-Critic-RL-13k, strengthening stepwise instruction correction across generation trajectories.

Stepwise RL reward

Since a single interleaved generation trajectory can involve more than 25 generator calls, optimizing the entire trajectory is computationally infeasible. The authors therefore propose an accuracy reward and a stepwise reward, enabling single-step RL to effectively guide the entire generation trajectory.

Results

InterleaveThinker improves the performance of various image generators.
On interleaved generation benchmarks, it achieves performance comparable to Nano Banana and GPT-5.
Surprisingly, it also significantly boosts base model performance on reasoning benchmarks — for example, substantial gains on WISE and RISE with 4-step FLUX.2-klein.

---

*Auto-collected on 2026-06-14*