InterleaveThinker: Adding Agentic Interleaved Text-Image Generation to Any Image Generator via a Planner-Critic-Generator Pipeline

Forum topic · 小凯 · 2026-06-14

Summary

InterleaveThinker is a multi-agent framework from CUHK MMLab and Meituan that enables any off-the-shelf image generator to perform interleaved text-image generation without retraining. The framework decouples planning, generation, and evaluation into three agents: a Planner that produces a complete multi-step plan before generation begins and never sees intermediate images, a Generator (e.g., FLUX.2-klein, Qwen-Image-Edit) that executes each step, and a Critic that verifies outputs and retries with refined prompts. This design addresses two key failures of unified multimodal models: visual over-reliance on intermediate states and step-wise error accumulation. Training uses GRPO reinforcement learning with dual accuracy and step-wise rewards on three self-built datasets covering 8 domains and 75 subcategories. The system outperforms open-source unified multimodal models, approaches closed-source systems like Nano Banana and GPT-5, and unexpectedly boosts text reasoning benchmarks (WISE +55%, RISE +117%). Code and weights are released on GitHub and Hugging Face.

InterleaveThinker Deep Dive: Giving Any Image Generator "Interleaved Text-Image Generation" Superpowers via a Planner-Critic-Generator Multi-Agent Pipeline

> Paper: *InterleaveThinker: Reinforcing Agentic Interleaved Generation* > Authors: Dian Zheng, Harry Lee, Manyuan Zhang, et al. (CUHK MMLab & Meituan) > Paper: https://arxiv.org/abs/2606.13679 > Code: https://github.com/zhengdian1/InterleaveThinker > Model weights: https://huggingface.co/InterleaveThinker

1. The Problem: Why Can't Existing Image Generators Do Interleaved Generation?

Single-image generation (SD/FLUX/Midjourney) is already strong, but real-world demands go beyond one image:

Visual storytelling: "Draw a 10-step story of a child learning to ride a bike"
Embodied manipulation: "A robot moves an apple from the table to the basket, with an image for each step"
Tutorials: "How to pan-sear a steak, shown step by step with text"

These require interleaved generation — taking a text-image sequence as input and outputting a coherent multi-step interleaved sequence. Existing image generators (diffusion or autoregressive) can only output a single image, so they architecturally cannot support interleaving.

Unified Multimodal Models (UMMs, e.g., Show-o, Emu3) architecturally support interleaving, but suffer from two fatal problems:

1.1 Visual Over-reliance

When UMMs generate a sequence, each step depends heavily on the previous visual state. For a push-up tutorial, the model might halt at an intermediate pose — because that intermediate state *looks* like completion. It loses the global goal and gets misled by local visual feedback.

1.2 Step-wise Error Accumulation

UMMs lack a reliable self-correction mechanism. Small early degradation in image quality gets progressively amplified, eventually ruining the entire output — the same mathematical issue as error accumulation in long-chain CoT, but in pixel space.

2. Core Idea: Three Agents with Clear Division of Labor

InterleaveThinker's solution is highly engineering-oriented: it does not modify the image generator itself, but wraps it in a multi-agent pipeline. The paper argues UMMs' two root problems stem from one model doing planning, generation, and evaluation simultaneously, letting visual state pollute planning.

The core idea: completely decouple planning, generation, and evaluation.

2.1 Three-Module Architecture (Figure 3)

Planner:

Input: the user's text-image sequence S
Output: an N-step execution plan, each step containing:
Step instruction u_i (human-readable description)
Model-friendly initial prompt p_i (adapted from u_i)
Auxiliary text a_i (supplementary knowledge, e.g., "steak needs to rest 3 minutes")
Key property: the Planner emits the complete plan once, before generation begins, and never touches any intermediate generated image. This fundamentally cuts off visual contamination of planning.

Generator:

Any off-the-shelf image generation/editing model (FLUX.2-klein, Qwen-Image-Edit, etc.)
Receives the Planner's prompt plus the previous image, generates the current step
Unmodified — invoked purely as a tool

Critic:

Evaluates whether the Generator's output I_i matches the Planner's original instruction p_i
Outputs a binary verdict j_i (pass/fail) + a refined prompt r_i+1 (on failure) + reasoning R_i
On failure, the Generator retries with the new prompt until pass or max iterations T_max

2.2 Why This Cures Both UMM Diseases

Visual over-reliance: The Planner plans once and never sees intermediate images. It cannot be misled by local hallucinations like "the intermediate state looks like the final goal." The global goal stays locked in the initial plan.

Error accumulation: The Critic's per-step verification ensures every step matches the original instruction, retrying on failure — like adding brakes at each step so early small errors can't snowball.

3. Data Engineering: A Self-Built Pipeline with 8 Categories, 75 Subcategories

The biggest hurdle for multi-agent frameworks is data — no ready-made aligned data exists. InterleaveThinker built a complete pipeline (Figure 5):

3.1 Text Prompt Construction

8 main categories: robotics, visual storytelling, art, workflows, daily life, science, professional skills, image captioning
75 subcategories (biology, cooking, physics, etc.)
Domain-specific vocabulary libraries (30+ terms) generated with Gemini 2.5 Pro
Filled into 100+ predefined templates ("How to {Action}", "Show {Action} step by step")
Result: ~40,000 diverse text prompts

3.2 Multi-Agent Trajectory Generation

Trajectories generated with Gemini 2.5 Pro and Nano Banana Pro
Per task: Planner generates global step instructions → Generator executes step by step → Critic evaluates and corrects if needed
Since Nano Banana Pro trajectories are extremely high quality, FLUX.2-klein-9B was added to balance the dataset and prevent the Critic from overfitting to a specific generator
Strict filtering pipeline for quality

3.3 Three Datasets

Interleave-Planner-SFT-80k: Planner training data for cold-start formatting
Interleave-Critic-SFT-112k: Critic supervised data for judging and correction
Interleave-Critic-RL-13k: Critic RL data, optimized with GRPO for step-wise correction

4. Training: Dual-Reward GRPO, Single-Step Optimization Achieving Trajectory-Level Alignment

An interleaved generation trajectory can involve 25+ generator calls. End-to-end trajectory optimization is computationally prohibitive.

InterleaveThinker uses a dual-reward strategy:

4.1 Reward Design

Accuracy Reward

Step-wise Reward

GRPO (Group Relative Policy Optimization) performs single-step RL, achieving trajectory-level alignment via single-step optimization, greatly reducing compute cost.

4.2 Why It Works

5. Results: Strong Generality, Plus Unexpected Reasoning Gains

5.1 Interleaved Generation Benchmarks

Outperforms all existing open-source UMMs
Performance approaches closed-source Nano Banana and GPT-5
Consistent gains when adapted to multiple generators (FLUX.2-klein, Qwen-Image-Edit, etc.)

5.2 Reasoning Benchmarks (Unexpected Bonus)

pure text reasoning benchmarks

WISE: 0.47 → 0.73 (+55%)
RISE: 13.3 → 28.9 (+117%)

The insight: the Planner-Critic collaboration trains long-horizon planning and step-wise verification, which generalizes from interleaved generation to pure text reasoning.

6. Why This Project Matters

6.1 Elegant Architectural Decoupling

Any generator can be upgraded — FLUX.2 today, Midjourney tomorrow, SANA later
Generators focus on what they're good at (high-quality single images) instead of being forced to learn planning
Planning and verification iterate independently — swap in a stronger LLM for the Planner, a sharper VLM for the Critic

6.2 Data Flywheel Potential

6.3 A Practical Multi-Agent Exemplar

practical exemplar with clear division of labor, a data closed-loop, and RL optimization

7. Limitations

1. Latency: multi-round Planner + Critic + Generator calls increase end-to-end latency. Even with fast 4-step FLUX.2-klein, long trajectories (25+ steps) remain slow 2. Data dependence: trajectory generation still relies on strong models like Gemini 2.5 Pro and Nano Banana Pro — costly 3. Iteration ceiling: T_max limits Critic retries; if the Generator itself is too weak, correction can't save it

8. Conclusion

InterleaveThinker is not yet another image generation model — it introduces agentic thinking to image generation. It proves a key proposition: existing strong image generators don't need retraining to do interleaved generation; they just need an agent that plans and an agent that quality-checks.

The idea's value goes beyond interleaving itself: it demonstrates a "plug-in agent upgrade" paradigm — keep the base model frozen and endow it with new capabilities via multi-agent collaboration.

---

References:

Zheng, D., et al. (2026). InterleaveThinker: Reinforcing Agentic Interleaved Generation. *arXiv preprint arXiv:2606.13679*.