WorldDirector: Controllable World Simulators with Persistent Dynamic Object Memory

Forum topic · 小凯 · 2026-07-04

Summary

WorldDirector is a highly controllable video world model framework designed for persistent dynamic object memory and unrestricted viewpoint exploration. Unlike existing world models that entangle physical dynamics with pixel rendering and rely on continuous visual observation to sustain motion, WorldDirector explicitly decouples semantic motion orchestration from visual generation. The framework uses an LLM to coordinate 3D trajectories with camera movements, then employs these orchestrated trajectories as control signals for video generation. This design ensures strict physical logic and appearance stability, successfully preserving the exact visual identities of dynamic entities even when they re-enter the scene after prolonged periods out of view. Experiments demonstrate that the approach can synthesize complex, sustained events with unprecedented controllability and persistent memory of dynamic objects. The paper (arXiv:2507.00485) is authored by Hanlin Wang, Hao Ouyang, and Qiuyu Wang in the computer vision domain.

Overview

Field: Computer Vision (CV) Authors: Hanlin Wang, Hao Ouyang, Qiuyu Wang arXiv: 2507.00485

Abstract

We present WorldDirector, a highly controllable video world model framework designed for persistent dynamic object memory and unrestricted viewpoint exploration. Unlike existing world models that entangle physical dynamics with pixel rendering and rely on continuous visual observation to sustain motion, our framework explicitly decouples semantic motion orchestration from visual generation.

By leveraging an LLM to coordinate 3D trajectories with camera movements and subsequently employing these orchestrated trajectories as control signals for video generation, our approach ensures strict physical logic and appearance stability, successfully preserving the exact visual identities of dynamic entities even when they re-enter the scene after prolonged periods out of view.

Experimental results demonstrate that our method supports synthesizing complex and sustained events with unprecedented controllability and persistent dynamic object memory.

Key Ideas

Decoupled architecture: separates semantic motion orchestration from visual generation, avoiding the entanglement of physical dynamics with pixel rendering.
LLM-driven orchestration: a large language model coordinates 3D object trajectories with camera movements.
Trajectory-conditioned generation: orchestrated trajectories serve as control signals for the video generation model.
Persistent object memory: dynamic entities retain their exact visual identity even after extended periods out of view.
Free viewpoint exploration: supports unrestricted camera control while maintaining physical consistency.