Jumpy World Models: Composing Sprinter Policies into a Marathon Runner

Forum topic · 小凯 · 2026-05-30

Summary

Researchers from McGill University, Meta FAIR, and Mila propose CompPlan, a test-time compositional planning method built on Jumpy World Models (arXiv:2602.19634). Instead of planning step-by-step with single-step dynamics—where model errors accumulate over long horizons—the approach treats pretrained policies as composable 'sprinter experts' and predicts, at the policy level, where each expert will end up using successor-measure flow matching across multiple time scales. A Temporal Difference flow-consistency loss (td-hc) aligns predictions across discount factors, and random shooting selects optimal policy sequences, replanning only after each policy's natural execution duration. On OGBench (antmaze navigation, multi-cube manipulation), CompPlan improves long-horizon performance by 201% relative to action-level planning and 89% over generalized policy improvement, reaching 90% on antmaze-giant and 67% on cube-4—surpassing hierarchical methods like HIQL and SHARSA without task-specific training. Weak individual policies become valuable as chain components, though the method assumes a sufficient policy library and a provided reward function.

Jumpy World Models: Composing "Sprinter Experts" into a "Marathon Runner"

In one sentence: A team from McGill University, Meta FAIR, and Mila enables robots to stop learning every long task from scratch—pretrained policies become composable "sprinter experts," jumpy world models predict where these experts will end up, and random shooting selects the optimal sequence of experts. Zero-shot performance improves by 200% on average.

| Item | Detail | |------|--------| | Paper | Compositional Planning with Jumpy World Models | | arXiv | 2602.19634v1 | | Date | February 23, 2026 | | Team | Jesse Farebrother (McGill/Mila), Matteo Pirotta (Meta FAIR), Andrea Tirinzoni (Meta FAIR), Marc G. Bellemare (McGill/Mila), Alessandro Lazaric (Meta FAIR), Ahmed Touati (Meta FAIR) | | Method | CompPlan: policy compositional planning + Jumpy World Models + temporal difference flow consistency (td-hc) | | Foundation | Temporal Difference Flows (Farebrother et al., 2025) — successor measure flow matching | | Benchmark | OGBench (antmaze medium/large/giant navigation; multi-cube pick-and-place manipulation) | | Key results | +201% over action-level planning, +89% over GPI on long-horizon tasks | | Policy base | Five policy families: GC-TD3, GC-1S, CRL, GC-BC, HFBC | | Planning mode | Test-time composition; no task-specific training or environment interaction |

Key points

The core problem: single-step model-predictive control suffers from error accumulation and exponential search growth over long horizons. The paper's insight: don't plan over individual actions—plan over policies.
Geometric Switching Policies (GSP): pretrained policies are chained, each executing for a geometrically distributed duration (switch probability α). The final policy is absorbing (α=0). Its successor measure decomposes into a weighted mixture of component policies' successor measures (Theorem 1).
Jumpy World Models (JWM): instead of predicting one-step dynamics, the model predicts the state distribution reached after running a policy π until a geometric stopping time at discount γ—learned via flow matching on the successor measure (Temporal Difference Flows).
td-hc consistency loss: predictions across time scales (short γ≈0.9, long γ≈0.99) are made self-consistent—long-horizon predictions must match "short jump, then long jump" compositions. Applied to only 25% (antmaze) or 12.5% (cube) of batches to avoid error amplification from self-sampling.
Planning algorithm: random shooting over candidate policy/goal sequences, one-sample Monte Carlo Q-estimation, then replanning after each policy's natural duration—reducing replanning frequency by a factor of K while expanding the effective planning horizon K-fold.

Results

Tested on OGBench with five policy families (HFBC, CRL, GC-TD3, GC-1S, GC-BC):

antmaze-giant: success rate jumps from ~10% to 90% in extreme cases
cube-4: from <10% to 67%
Average long-horizon improvement: +201% vs. action-level planning, +89% vs. GPI
Weak zero-shot policies become strong in composition: GC-BC scores near 0% zero-shot on cube-4 but exceeds 70% within CompPlan—composability cannot be judged from individual policy performance
Outperforms train-time hierarchical methods: HIQL achieves 0% vs. CompPlan's 67% on cube-4; SHARSA achieves 9% vs. 67%

Honest limitations noted by the author

Replanning compute cost: wall-clock comparisons are missing; feasibility on real robots is unaddressed.
Policy diversity assumption: CompPlan cannot invent missing skills—it can only recombine what the library contains.
Geometric stopping is a mathematical convenience, not physical reality; the mapping between the GSP abstraction and deterministic replanning isn't deeply discussed.
Reward source unresolved: the method optimizes given rewards but doesn't address where rewards come from.
Pretraining cost is not counted in the "zero-shot" claim.

Commentary

The most exciting reframing: traditional world models are simulators of physics (state + action → next state); jumpy world models are simulators of behavior (policy + time scale → where you end up). The space of policy combinations is far smaller than the space of action sequences. The paper builds a complete pipeline from flow matching to decision-making, each step theoretically grounded. A promising future direction: if policies themselves are generated (e.g., diffusion policies), compositional planning becomes a closed loop of search + generation + evaluation over policy space.

References: 1. Farebrother et al., "Compositional Planning with Jumpy World Models", arXiv:2602.19634, 2026. 2. Farebrother et al., "Temporal Difference Flows", arXiv:2503.09817, 2025. 3. Thakoor et al., "Geometric Generalized Policy Improvement", 2022. 4. Park et al., "OGBench: Benchmarking Offline Goal-Conditioned RL", 2025. 5. Janner et al., "Gamma Models", 2020.