InterleaveThinker Explained: Giving Any Image Generator "Interleaved Image-Text Generation" Superpowers via a Planner-Critic-Generator Multi-Agent Pipeline

> Paper: *InterleaveThinker: Reinforcing Agentic Interleaved Generation* > Authors: Dian Zheng, Harry Lee, Manyuan Zhang, et al. (CUHK MMLab & Meituan) > Paper: https://arxiv.org/abs/2606.13679 > Code: https://github.com/zhengdian1/InterleaveThinker > Model weights: https://huggingface.co/InterleaveThinker

1. The Problem: Why Existing Image Generators Can't Do Interleaved Generation

Single-image generation (SD/FLUX/Midjourney) is already strong, but real-world demands go far beyond one image. Examples include:

• Visual storytelling: "Draw a 10-step story of a kid learning to ride a bike"
• Embodied manipulation: "Move an apple from the table to a basket, with an image for each step"
• Tutorials: "How to pan-sear a steak, alternating text and images"

These tasks require interleaved generation — taking an image-text sequence as input and outputting a coherent multi-step image-text sequence. But existing image generators (diffusion / autoregressive models) can only output a single image by design.

Unified Multimodal Models (UMMs, e.g., Show-o, Emu3) architecturally support interleaved generation but suffer from two critical flaws:

1.1 Visual Over-reliance

UMMs depend heavily on the previous step's visual state at each generation step. For a push-up tutorial, the model may stall at an intermediate pose — because that intermediate state visually looks "complete." It loses the global goal and gets misled by local visual feedback.

1.2 Step-wise Error Accumulation

UMMs lack a reliable self-correction mechanism. Small early quality degradations amplify step by step until the entire output is ruined — the same mathematical problem as error accumulation in long chain-of-thought reasoning, but happening in pixel space.

2. Core Idea: Three Agents with Clear Division of Labor

InterleaveThinker's solution is highly engineering-driven: don't modify the image generator — wrap it in a multi-agent pipeline. The authors argue that both UMM pathologies (visual dependence + error accumulation) stem from a single model doing planning, generation, and evaluation simultaneously, letting visual states pollute planning.

The core idea: completely decouple planning, generation, and evaluation.

2.1 Three-Module Architecture (Figure 3)

Planner:

• Input: user's image-text sequence S
• Output: an N-step execution plan, each step containing:
• Step instruction u_i (human-readable description)
• Model-friendly initial prompt p_i (adapted from u_i)
• Auxiliary text a_i (supplementary knowledge, e.g., "the steak needs to rest 3 minutes")
• Key point: the Planner outputs the complete plan in one shot before generation starts, and never touches any intermediate generated images. This fundamentally cuts off visual feedback pollution of planning.

Generator:

• Any off-the-shelf image generation/editing model (FLUX.2-klein, Qwen-image-Edit, etc.)
• Receives the Planner's prompt and the previous image, generates the current step's image
• Unmodified — invoked purely as a tool

Critic:

• Evaluates whether the Generator's output I_i matches the Planner's original instruction p_i
• Outputs a binary judgment j_i (pass/fail) + a refined prompt r_i+1 (when failing) + reasoning R_i
• On failure, the Generator retries with the new prompt until passing or reaching the max iteration limit T_max

2.2 Why This Cures Both UMM Pathologies

Visual over-reliance: The Planner plans once, upfront, never seeing intermediate images. It cannot be misled by local hallucinations like "the intermediate state looks like the final goal." The global goal stays locked in the initial plan.

Error accumulation: The Critic's per-step verification ensures every output matches the original instruction, retrying on failure — like adding brakes at each step so early small errors never snowball.

3. Data Engineering: A Self-Built Pipeline with 8 Categories and 75 Subcategories

The biggest hurdle for multi-agent frameworks is data — no ready-made alignment data exists. InterleaveThinker built a complete data pipeline (Figure 5):

3.1 Text Prompt Construction

• 8 major categories: robotics, visual storytelling, art, workflows, daily life, science, professional skills, image captioning
• 75 subcategories (e.g., biology, cooking, physics)
• 30+ domain-specific vocabulary lists generated with Gemini 2.5 Pro
• 100+ predefined templates filled in ("How to {Action}", "Show {Action} step by step")
• ~40,000 diverse text prompts produced

3.2 Multi-Agent Trajectory Generation

• Trajectories generated with Gemini 2.5 Pro and Nano Banana Pro
• Per task: Planner generates global step instructions → Generator executes step by step → Critic evaluates and corrects as needed
• Since Nano Banana Pro trajectories are extremely high quality, FLUX.2-klein-9B was added to balance the dataset and prevent Critic overfitting to one generator
• Strict filtering pipeline ensures quality

3.3 Three Datasets

• Interleave-Planner-SFT-80k: Planner SFT data for format cold-start
• Interleave-Critic-SFT-112k: Critic supervised data for judgment and correction
• Interleave-Critic-RL-13k: Critic RL data, optimized with GRPO for step-wise correction

4. Training: Dual-Reward GRPO, Trajectory-Level Alignment via Single-Step Optimization

An interleaved generation trajectory can involve 25+ generator calls. End-to-end optimization of the whole trajectory is computationally infeasible.

InterleaveThinker solves this with a dual-reward strategy:

4.1 Reward Design

1. Accuracy Reward: alignment of the final output with the target (e.g., task completion) 2. Step-wise Reward: per-step correction quality (whether the Critic's prompt refinement worked and the retry passed)

Using GRPO (Group Relative Policy Optimization) for single-step RL, single-step optimization achieves trajectory-level alignment, dramatically reducing compute cost.

4.2 Why It Works

GRPO's key advantage is requiring no critic network — advantages are estimated from group-relative rewards. For the Critic's step-wise correction task, each step's reward is computed independently without waiting for the full trajectory, enabling per-step parallel training and greatly reducing long-trajectory optimization overhead.

5. Results: Full Generality, Plus Unexpected Reasoning Gains

5.1 Interleaved Generation Benchmarks

Using 4-step FLUX.2-klein as the generator:
• Outperforms all existing open-source UMMs
• Performance approaches closed-source Nano Banana and GPT-5
• Adapts to multiple generators (FLUX.2-klein, Qwen-image-Edit, etc.) with consistent gains

5.2 Reasoning Benchmarks (Bonus)

Surprisingly, InterleaveThinker also improves substantially on pure text reasoning benchmarks:

• WISE: 0.47 → 0.73 (+55%)
• RISE: 13.3 → 28.9 (+117%)

The insight: the Planner-Critic synergy essentially trains long-horizon planning and step-wise verification abilities, which generalize from interleaved generation to pure text reasoning.

6. Why This Project Matters

6.1 Elegant Architectural Decoupling

InterleaveThinker doesn't touch the generator's weights — it just wraps the Planner and Critic around it. This means:
• Any generator can be upgraded — FLUX.2 today, Midjourney tomorrow, SANA after
• Generators focus on what they do best (high-quality single images) instead of being forced to learn planning
• Planning and verification iterate independently — swap in a stronger LLM Planner or a sharper VLM Critic anytime

6.2 Data Flywheel Potential

The self-built data pipeline is a long-term moat: 40k+ diverse prompts, 80k+ Planner examples, 110k+ Critic examples, 13k RL examples, covering 75 subcategories from robotics to cooking — giving the framework strong zero-shot generalization to new interleaved task types.

6.3 A Practical Multi-Agent Exemplar

"Multi-agent" is often concept hype in agent system design. InterleaveThinker delivers a practical exemplar with clear division of labor, a data loop, and RL optimization. The Planner-Critic-Generator decoupling pattern transfers to any "plan + execute + verify" agent scenario.

7. Limitations

1. Latency: Planner + Critic + Generator multi-round calls increase end-to-end latency. 4-step FLUX.2-klein is fast, but long trajectories (25+ steps) remain slow 2. Data dependence: high-quality trajectory generation still relies on strong models like Gemini 2.5 Pro and Nano Banana Pro — not cheap 3. Iteration cap: T_max limits Critic retries; if the Generator itself lacks capability, correction may not save the output

8. Conclusion

InterleaveThinker is not another image generation model — it brings agentic thinking to image generation. It proves a key thesis: existing strong image generators don't need retraining for interleaved generation; they just need an agent that plans and an agent that quality-checks.

The value goes beyond interleaved generation — it demonstrates a "plug-in agent upgrade" paradigm: keep the base model frozen, and use multi-agent collaboration to grant it new capabilities.

---

References:

• Zheng, D., et al. (2026). InterleaveThinker: Reinforcing Agentic Interleaved Generation. *arXiv preprint arXiv:2606.13679*.