Jumpy World Models: Composing "Sprinter Experts" into "Marathon Runners"
> TL;DR: A team from McGill University, Meta FAIR, and Mila shows that robots no longer need to learn every long-horizon task from scratch. By treating pre-trained policies as composable "sprinter experts," predicting with a *jumpy world model* where each expert will end up, and using random shooting to select the best sequence, zero-shot performance improves by ~200% on average.
| Item | Detail | |------|--------| | Paper | Compositional Planning with Jumpy World Models | | arXiv | arXiv:2602.19634v1 | | Date | February 23, 2026 | | Team | Jesse Farebrother (McGill/Mila), Matteo Pirotta (Meta FAIR), Andrea Tirinzoni (Meta FAIR), Marc G. Bellemare (McGill/Mila), Alessandro Lazaric (Meta FAIR), Ahmed Touati (Meta FAIR) | | Core method | CompPlan: policy-compositional planning + Jumpy World Models (JWM) + temporal-difference flow consistency (td-hc) | | Foundation | Temporal Difference Flows (Farebrother et al., 2025), successor-measure flow matching | | Benchmarks | OGBench (navigation: antmaze medium/large/giant; manipulation: multi-cube pick-and-place) | | Key results | +201% over action-level planning and +89% over GPI on long-horizon tasks | | Policy backbones | Five policy families: GC-TD3, GC-1S, CRL, GC-BC, HFBC | | Planning | Test-time composition; no task-specific training or environment interaction |
The core problem: the error hell of single-step planning
Imagine a robot escaping a giant maze. Traditional model predictive control (MPC) replans every step: observe state, predict the next state, choose an action, execute, repeat.
The problem is error accumulation. Each one-step prediction has small bias; after a hundred steps, the robot may have hit walls repeatedly. Worse, planning complexity explodes exponentially with horizon—each extra step multiplies the search space by the action-space cardinality.
Long-horizon tasks have thus become a graveyard for RL: model-based RL excels at short horizons but collapses on long ones.
The paper's insight: don't plan over single-step actions—plan over policies.
Three core building blocks
1. Geometric policy switching (GSP) — policies as building blocks
The paper does not learn new policies. It assumes a library of pre-trained policies, each a "sprint expert" good at a subtask (walk toward goal g, pick up the red cube, avoid obstacles).
How do you chain experts into a pipeline that solves a task none of them can solve alone?
Using the geometric policy composition framework of Thakoor et al. (2022), each policy executes for a geometrically distributed duration—with probability α it switches to the next expert, with probability 1−α it keeps running. The final policy is an absorbing state with α=0.
The resulting non-Markovian policy is a geometric switching policy (GSP). Mathematically, its successor measure decomposes into a weighted mixture of the component policies' successor measures—the core of Theorem 1.
Key insight: since the GSP successor measure is a linear combination of component measures, predicting each policy's successor distribution at different timescales suffices to evaluate any policy sequence.
2. Jumpy world models (JWM) — flow matching across timescales
Traditional world models predict "given state s and action a, where is s'?"—one-step dynamics.
A jumpy world model instead asks: "given policy π and discount γ, where does the system end up after running π until a geometrically distributed stopping time?"
That is a multi-step, policy-level, distributional prediction, corresponding to the successor measure m^π_γ(·|s,a)—the state-occupancy distribution induced by π under geometric discount.
Temporal Difference Flows (Farebrother et al., 2025) learn this via flow matching: model the successor measure as an ODE-driven probability path from prior noise X₀ through a vector field v_t, reaching m^π_γ at t=1.
The td-flow training loss mixes two targets:
- One-step term: flow-matching target pointing at the immediate transition kernel P(·|s,a), weight (1−γ)
- Bootstrap term: target pointing at the successor measure itself, weight γ
- HFBC (hierarchical flow behavior cloning) is most robust, ~10% success even on giant mazes and cube-4
- CRL (contrastive RL) is strong on antmaze but weak on cube—its Q(s,a,g) ≈ φ(s,a)ᵀψ(g) factorization is spatially sensitive but insensitive to object-manipulation features
- GC-TD3, GC-1S, GC-BC do okay at short horizons and collapse on long ones (cube-4 success <15%)
- antmaze-giant: 10% → 90% (extreme case)
- cube-4: <10% → 67%
- On average, long-horizon tasks improve 201% relative to action-level planning (ActionPlan)
- Replanning compute: policy-level planning reduces replanning frequency by K², but no wall-clock numbers are given. Is GHM inference + random shooting latency acceptable on real robots (OGBench is MuJoCo simulation)? Unstated.
- Policy diversity assumption: composition requires the library to cover needed sub-skills. If a new task needs a skill absent from the library, CompPlan cannot invent it—it can only recombine existing ones. This tensions with the "zero-shot" label.
- Physical meaning of γ: the geometric stopping time is a mathematical convenience, not physical reality—robots don't switch policies by coin flips. GSP is an abstraction; actual execution is deterministic replanning. Whether this mapping always holds is not deeply discussed.
- Reward assumption: evaluating GSP value requires r(s). Where rewards come from in real settings is unsolved.
- Consistency sample efficiency: the 12.5–25% consistency-batch ratio is hand-tuned; no sensitivity analysis or exploration of an irreducible long-horizon error floor for flow matching.
- Contributions: (1) jumpy world models learning policy-level multi-step dynamics; (2) the GSP successor-measure decomposition theorem, turning compositional policy evaluation into a weighted mixture of component evaluations; (3) the td-hc consistency loss for stable cross-timescale bootstrapping; (4) CompPlan, a test-time compositional planner beating hierarchical SOTA without task-specific training.
- Limitations: replanning compute unquantified; policy-library coverage assumption unchallenged; γ abstraction vs. physical execution mapping unexplored; reward provenance unsolved; pre-training costs excluded from the "zero-shot" accounting.
This mixture exactly matches the successor measure's Bellman equation: m^π_γ = (1−γ)P + γ E[m^π_γ(·|S',A')].
3. Temporal-difference flow consistency (td-hc) — locking timescales together
A single timescale is not enough: planning needs short jumps (γ≈0.9, tens of steps) and long jumps (γ≈0.99, hundreds of steps).
Training separate models per scale is wasteful and unstable. The paper proposes cross-timescale consistency: long-horizon predictions should decompose self-consistently as "a short jump, then a long jump."
Using Theorem 1, m^π_γ can be decomposed through m^π_β (β<γ), yielding the td-hc loss: an added consistency term requiring that the γ-scale prediction matches the composition of a β-scale jump to an intermediate state followed by a γ-scale jump from there.
In practice, only 25% (antmaze) or 12.5% (cube) of batches use the consistency term—because consistency requires sampling the model's own predictions as intermediate conditions, and errors self-amplify. A small fraction of consistency updates provides cross-scale alignment; most updates remain clean td-flow.
Planning: random shooting + policy-level replanning
With the GHM (Geometric Horizon Model), planning becomes optimizing:
max over z₁,...,zₙ of Q^ν_γ(s, a₁),
where ν = π_{z₁} → π_{z₂} → ... → π_{zₙ} is a GSP with geometric switching.
The optimization is plain random shooting:
1. Proposal distribution: use the GHM itself to generate candidate subgoal sequences. Given goal g and state s, sample actions and next subgoals with goal-conditioned policies, recursively producing (z₁,...,zₙ). For multi-path tasks like cube, also sample 1024 candidate sequences with the unconditional GHM. 2. Value evaluation: score each candidate with a one-sample Monte Carlo estimator (Lemma 1), sampling a trajectory through all policy components and weighting per-stage rewards by w_k. 3. Selection and execution: pick the highest-Q sequence, execute the first action a*₁, follow policy π_{z*₁}, and replan upon reaching the new state.
Replanning happens after each policy's natural episode ends—not every step. This reduces replanning frequency by a factor of K (the average policy length) while extending the effective planning horizon by K.
Experiments: where the 200% comes from
Baseline policies' zero-shot performance
On OGBench (antmaze medium/large/giant; cube manipulation with 1–4 cubes), the five baseline families are uneven:
The compositional leap
CompPlan improves every policy family across the board, most dramatically on long-horizon tasks:
The most striking finding: policies with poor zero-shot performance compose well. GC-BC is nearly 0% zero-shot on cube-4, yet reaches 70%+ inside CompPlan. Compositional value cannot be judged from zero-shot performance—a weak policy may still be a valuable link in a chain.
Comparison with other planning methods
| Method | Nature | Relative to CompPlan | |--------|--------|----------------------| | GPI (generalized policy improvement) | picks the best single policy each step, no composition | −89% | | ActionPlan | single-step action-level MPC | −201% | | HIQL (hierarchical implicit Q-learning) | train-time hierarchy | 0% vs 67% on cube-4 | | SHARSA (current hierarchical SOTA) | train-time hierarchy | 9% vs 67% on cube-4 |
CompPlan not only beats action-level planning but crushes task-trained hierarchical methods—a win for test-time planning over train-time hierarchies.
Honest unknowns
My take
The most exciting part is not the 200% number—it's the reframing of what a world model is for.
Traditional world models are "simulators of the environment": given state and action, predict the next state. Jumpy world models are "simulators of policies": given a policy and timescale, predict where it takes you. Once policies are pre-trained, parameterized, and indexable (z ∈ Z), the world model upgrades from predicting physics to predicting behavior—and the compositional space of behaviors is far smaller than that of action sequences.
Deeper still, the paper presents a complete pipeline from flow matching to decision-making: TD Flows → GSP decomposition theorem → td-hc consistency → test-time random shooting. Every link has theoretical support.
But I remain cautious about the "no task-specific training" claim. CompPlan doesn't retrain networks per task, but it does require a pre-trained policy library and a pre-trained GHM—costs not counted in the "zero-shot" accounting.
A direction worth watching: what if policies themselves were generative? If z could be sampled by a generative model (e.g., a diffusion policy), compositional planning would upgrade into a closed loop of search + generation + evaluation over policy space—a more radical paradigm.