WorldGuide: A Goal-Directed Video World Model for Procedural Task Execution

Forum topic · 小凯 · 2026-10-10

Summary

WorldGuide (arXiv:2610.12459) is a closed-loop video world model for long-horizon procedural tasks. Instead of open-loop generation, it iteratively predicts an atomic action from the current generated state, synthesizes the corresponding video clip, and uses the outcome to decide the next action or detect task completion. A Planner and an Executor are trained on the same step-level procedural demonstrations, with a hierarchical visual memory maintaining long-range state under bounded token costs. Since step-level action-video supervision was lacking, the authors introduce WorldGuide Bench: roughly 59K step-annotated videos covering 245 tasks across 27 procedural categories. WorldGuide achieves 33.33% task success on WorldGuide-Bench, surpassing MiniMax-H3 (29.90%) even when the latter receives a reference action plan, and reaches 47.69% on VideoCraft-Bench with only a goal (vs. 32.73% for MiniMax-H3). The results show that coupling planning with learned execution is essential for goal-directed procedural video generation.

Paper Overview

Field: Computer Vision Authors: Ankan Deria, Komal Kumar, Hisham Cholakkal, Fahad Shahbaz Khan, Salman Khan Published: 2026-10-08 arXiv: 2610.12459

Summary (translated)

Video generators and video-based world models can synthesize plausible visual trajectories, but long-horizon procedural tasks require generation to adapt to what has actually been produced: the model must decide the next action from its generated state, execute that action, and recognize when the task is complete. Open-loop generation cannot adapt to execution outcomes, while existing closed-loop systems often rely on pretrained executors or indirect verification — leaving a gap between deciding an action and successfully realizing it.

The authors formulate procedural video generation as closed-loop task execution in visual world space and introduce WorldGuide. Given an initial image and a task goal, WorldGuide predicts an atomic action, generates its corresponding video clip, then uses the generated result to select the next action or terminate.

Planner and Executor are trained on the same step-level procedural demonstrations: the Planner learns to predict the next atomic action or task completion from visual progress, while the Executor is trained directly to realize the predicted action.
A hierarchical visual memory maintains long-range execution state with a bounded history-token cost.
Due to the absence of step-level action-video supervision, they introduce WorldGuide Bench: ~59K step-annotated videos covering 245 tasks across 27 procedural categories.

Results

| Benchmark | Model | Success Rate | |---|---|---| | WorldGuide-Bench | WorldGuide | 33.33% | | WorldGuide-Bench | MiniMax-H3 (with reference action plan) | 29.90% | | VideoCraft-Bench (goal only) | WorldGuide | 47.69% | | VideoCraft-Bench (goal only) | MiniMax-H3 | 32.73% |

These results demonstrate that coupling planning with learned execution is critical for goal-directed procedural video generation.

Original Abstract (excerpt)

> Video generators and video-based world models can synthesize plausible visual trajectories, but long-horizon procedural tasks require generation to adapt to what has actually been produced. A model must determine the next action from its generated state, execute that action, and recognize when the task is complete. Open-loop generation cannot adapt to execution outcomes, while existing closed-loop systems often rely on pretrained executors or indirect verification. This leaves a gap between deciding an action and successfully realizing it. We formulate procedural video generation as *closed-loop task execution in visual world space* and introduce WorldGuide...

Full paper: <https://arxiv.org/abs/2610.12459>