SANA-WM: NVIDIA Fits a 'One-Minute World Model' onto a Single GPU
> Core finding: 2.6B parameters, 15 days of training on 64 H100s, single-H100 inference — NVIDIA combines "hybrid attention + dual-branch camera control + two-stage refinement" to turn 720p one-minute video generation from an "8-GPU-cluster exclusive" into something a single GPU can handle. Throughput is 36x higher than the 14B+14B LingBot-World, with better camera-control accuracy. This is a victory of engineering efficiency, not parameter stacking.
One-line summary
NVIDIA released SANA-WM — a 2.6B-parameter open world model that turns one image and a camera trajectory into controllable 720p, 60-second video. Training took 64 H100s for 15 days; inference runs on a single H100. A distilled version generates a full minute in 34 seconds on an RTX 5090. Before it, comparable systems needed a 14B+14B dual-model setup and an 8-GPU cluster with throughput of only 0.6 videos/hour. SANA-WM pushes that to 24.1 — not by being bigger, but by using smarter attention.
Why "one minute" is hard
Most video generation models stall at 5–10 seconds because standard softmax attention has quadratic memory and compute complexity in sequence length. 720p at 16 fps for 60 seconds means 961 latent frames — full softmax attention OOMs on that. Prior work either drops to 480p, uses 8-GPU clusters, or both. SANA-WM's goal: keep 720p, run on one GPU, train natively at one minute.
Four engineering designs
1. Hybrid linear attention: Gated DeltaNet + softmax
Pure linear attention lacks a decay mechanism: all history accumulates into the recurrent state with equal weight, causing state drift at minute scale (the problem of SANA-Video's cumulative ReLU-based linear attention).
Gated DeltaNet (GDN) adds two control valves:
- γ decay gate: multiplies the previous state by a learned decay scalar each step, fading stale content.
- β delta-rule correction: updates only the residual between prediction and target instead of overwriting the state.
- Coarse branch: UCPE (Unified Camera Pose Embedding) encodes global 6-DoF pose at latent-frame rate, building the coordinate skeleton.
- Fine branch: Plücker Raymap captures high-frequency in-frame motion at raw-frame rate.
- ΔIQ (first-10s vs last-10s quality gap) before refinement: 3.79/3.09; after: 1.17/0.31
- HY-WorldPlay's ΔIQ: 23.59/25.88 — late-segment collapse
- Stage 1 (~2.75 days): adapt pretrained SANA-Video to the frame-wise GDN architecture on short (5s) clips to diagnose GDN failure modes.
- Stage 2 (~2 days): introduce hybrid attention — one softmax block per 4 GDN blocks; tune efficiency-quality trade-offs on short clips.
- Stage 3 (~8 days): scale to 961-frame (60s) sequences and add dual-branch camera control. Context-Parallel (CP=2) shards the latent sequence; GDN transition matrices use prefix-sum composition — mathematically exact with minimal communication.
- Stage 4 (~2.5 days): fine-tune a chunk-causal autoregressive variant; self-forcing distillation compresses sampling to 4 denoising steps; softmax layers get attention-sink tokens and local temporal windows for constant-memory long rollouts.
- Autonomous driving simulation: a street-view image + planned path → continuous first-person driving video, without costly road capture.
- Robot training: a virtual world with realistic visuals and feedback for practice.
- Game content generation: one-click level flythroughs for previewing geometry and lighting.
- Film previsualization: shot-list video previews before actual filming.
- Paper: SANA-WM: Efficient Minute-Scale World Modeling with Hybrid Linear Diffusion Transformer (arXiv: 2605.15178)
- Authors: Haoyi Zhu, Haozhe Liu, Yuyang Zhao, Tian Ye, Junsong Chen, Jincheng Yu, Tong He, Song Han, Enze Xie
- Institution: NVIDIA
- Project page: https://nvlabs.github.io/Sana/WM/
- Code: https://github.com/NVlabs/Sana (SANA-WM code not yet released)
- Architecture: 20-block DiT, 15 GDN + 5 softmax blocks, d_model=2240, 20 heads
- Training: ~213K clips, 64 H100 × ~18.5 days (incl. VAE pre-adaptation)
- Inference: single H100 → 60s 720p; RTX 5090 + NVFP4 quantization + distilled → 34s
- VAE: LTX2-VAE, C=128 latent channels
The state stays a constant D×D size regardless of frame count. But pure recurrence is weak at long-range spatial recall, so SANA-WM interleaves 15 frame-wise GDN blocks with 5 softmax attention blocks (layers 3, 7, 11, 15, 19) across 20 transformer blocks.
Training stability trick: standard L2 key normalization triggers NaN at step 1, so the team uses algebraic key scaling — keys scaled by 1/√(D·S), where D is head dimension and S is spatial tokens per frame — locking the transition matrix's spectral norm.
Result: full softmax OOMs at 60s; pure GDN drifts; the hybrid gets both long-range precision and constant memory.
2. Dual-branch camera control
The branches are input independently and fused inside blocks. Result: RotErr 4.50°/8.34° (Simple/Hard), CamMC 1.41/1.44 — more accurate than 14B+14B LingBot-World.
3. Two-stage generation pipeline
Stage-1 produces a spatiotemporally consistent draft with structural flaws at long horizons. Stage-2 applies a 17B LTX-2 refiner with rank-384 LoRA adapters (Q/K/V/O and FFN projections), fine-tuned via truncated-σ flow matching with a large starting noise (σ_start=0.9). Inference needs only 3 Euler denoising steps; LoRA is merged into the base, so end-to-end throughput barely drops (24.1 → 22.0 videos/hr).
4. Robust annotation pipeline
Instead of 3D simulation or manual capture, SANA-WM extracts metric-scale 6-DoF camera poses directly from public videos — about 213K clips with metric-scale pose supervision, no synthetic data.
The numbers: where the 36x throughput comes from
| Metric | SANA-WM | LingBot-World | HY-WorldPlay | Matrix-Game 3.0 | Infinite-World | |--------|---------|---------------|--------------|-----------------|----------------| | Parameters | 2.6B | 14B+14B | 8B | 5B | 1.3B | | Resolution | 720p | 480p | 480p | 720p | 480p | | Inference GPUs | 1 | 8 | 8 | 8 | 1 | | Rot. error (°) Simple | 4.50 | 10.47 | 17.89 | 12.96 | 16.55 | | Rot. error (°) Hard | 8.34 | 18.99 | 35.46 | 18.79 | 41.31 | | VBench Overall Simple | 80.62 | 81.82 | 68.82 | 78.53 | 79.18 | | VBench Overall Hard | 81.89 | 81.89 | 70.46 | 78.79 | 79.51 | | Throughput (videos/hr) | 22.0 | 0.6 | 1.1 | 3.1 | 5.9 | | Memory (GB) | 74.7 | 454.1 | 215.5 | 106.2 | 53.5 |
Key insight: with 9% of the parameters (2.6B vs 28B) on 1/8 the inference GPUs, SANA-WM achieves equal-or-better camera control accuracy, similar visual quality, 36x higher throughput, and 1/6 the memory.
Training pipeline: four progressive stages
Total training: ~15 days on 64 H100s (VAE pre-adaptation adds ~3.5 days). Custom fused Triton kernels for GDN scan and gate operations give 1.5–2x throughput gains — efficiency lives in every line of kernel code, not just the architecture diagram.