🗺️ From 18% to 73% in a maze: LeCun's company delivers
A quadruped robot walking a maze reaches a fork: go left or right? That decision depends on position and corridors. But which leg to move in the next second depends on body posture and the ground beneath. The same robot, in the same second, makes two very different judgments requiring different information.
Most world models use a single set of features for both — and start to struggle as tasks get longer.
A paper submitted October 5 (arXiv:2610.06805, *H-JEPA: End-to-End Learning of Hierarchical World Models for Visual Planning*) takes the approach of explicit layering, with each level learning what it needs. The five authors are Wancong Zhang, Basile Terver, Michael Rabbat, Yann LeCun, and Randall Balestriero.
Behind LeCun's name is AMI (Advanced Machine Intelligence). In March this year the company announced $1.03 billion in funding at a $3.5 billion pre-money valuation. Rabbat, a co-author, leads world models at AMI.
🪜 One JEPA per layer, each predicting its own thing
JEPA stands for joint-embedding predictive architecture. While LLMs learn generation by predicting the next token, visual JEPAs encode an image into an internal representation and predict the representation of a target region — without generating pixels one by one. H-JEPA stacks multiple JEPAs layer by layer; each layer predicts over a different time span, and each layer predicts in its own representation space.
Planning runs top-down. The highest layer optimizes "progress toward the goal"; its predicted intermediate states become subgoals for the next layer, which refines them in turn, down to executable actions.
Each layer has three components: a state encoder extracting the current environment representation (the lowest layer reads images directly; higher layers abstract from lower-layer representations), an action encoder describing the change an action causes (concrete actions at low layers; coarser compression of action sequences at higher layers), and a predictor combining state and action to predict future state. In the paper's experiments, one step at a higher layer corresponds to two prediction steps at the layer below.
The researchers then examined what each layer retained. In the maze task, higher layers gradually dropped leg posture but preserved the robot's position — exactly the intended design: higher layers discard fast-changing detail and keep slow-changing, task-relevant state.
📈 The numbers: 18% to 73%, with less compute
The abstract's headline comparison is on the Visual AntMaze simulated maze: a three-layer H-JEPA raised success rate from 18% to 73% while using less planner compute. Chinese coverage reports this as 73.3% vs. 18.0%, adding that the three-layer model needs less planning compute.
The authors tested four environments, with LeWorldModel (LeWM) and HWM as baselines. The abstract mentions "across four simulated navigation and manipulation environments"; Chinese coverage names them as Visual AntMaze, FourRoom Distractors, Push-T, and OGBench Cube.
| Environment | Result | Source | |---|---|---| | Visual AntMaze | 18% → 73%, less planner compute | Stated explicitly in abstract | | FourRoom Distractors | Hierarchy beats flat JEPA | Mentioned in abstract and coverage, no numbers | | Push-T | Two layers best; three and four layers worse | Baseline comparison mentioned; numbers not given | | OGBench Cube | Hierarchy beats flat JEPA | Mentioned in abstract and coverage, no numbers |
Aside from the Visual AntMaze pair, no concrete success rates for the other three environments appear in the abstract or public coverage. This is the key caveat: the four-environment comparison is the paper's core experiment, yet only one environment has verifiable hard numbers.
On ablations, the abstract states: "Ablations attribute these gains to both temporal decomposition and higher-level goal representations." Post-ablation residual values were not provided.
🔧 More layers is not always better
Push-T is the only named counterexample. Chinese coverage is explicit: two layers performed best on Push-T, and performance dropped at three and four layers. The paper's own explanation is that shorter task trajectories may limit the training data available to higher layers.
That explanation is plausible — higher layers need to see enough long-horizon trajectories to learn long-horizon structure; if a task ends in three steps, higher layers mostly learn noise.
Note, however, that this is only one possible explanation offered by the paper; no experiment validating it appears in the abstract or coverage.
🤖 No real-robot deployment yet — offline data only
The abstract's final sentence mentions DROID: "With inverse-dynamics supervision, the approach extends to diverse real-robot videos from DROID, where hierarchy improves offline planning fidelity at lower planner compute."
Inverse-dynamics supervision, the DROID real-robot video dataset, improved offline planning fidelity, lower planner compute.
Offline. No closed-loop real-robot execution, no real-robot success rates. The abstract does not disguise this, and the paper does not mention real-robot deployment.
One more piece of technical background: H-JEPA uses SIGReg, proposed in LeJEPA, to constrain the representation distribution and prevent representation collapse (different states being compressed into the same vector). LeWorldModel applied this to world-model training this year, and H-JEPA stacks on top of it as the lowest layer.
The paper and pretrained weights are open-sourced and reproducible. This continues LeCun's open-source stance from his Meta era. Incidentally, at ECCV in Malmö, Sweden in September, his talk slides once again carried the advice: "If you're interested in human-level AI, don't work on LLMs." When some suspected the slide was fake, he replied: it's real.
⚖️ What this research does not say
- Preprint; peer review not completed. The abstract page shows no review status.
- Only Visual AntMaze has verifiable hard numbers; success rates for the other three environments, post-ablation values, and the specific planner-compute ratio were not obtained.
- All results come from simulation and offline evaluation; no closed-loop real-robot results exist.
- The cause of Push-T's degradation with more layers is only a conjecture in the paper.
- The paper's size (14,227 KB) suggests many figures, but the abstract gives no parameter counts, training data volume, or compute cost.
- Paper: Wancong Zhang, Basile Terver, Michael Rabbat, Yann LeCun, Randall Balestriero, *H-JEPA: End-to-End Learning of Hierarchical World Models for Visual Planning*, arXiv:2610.06805, submitted 2026-10-05 17:53 UTC, 14,227 KB, cs.LG / cs.RO.
- All figures (18%→73%, less planner compute, four environments, ablation attribution, DROID offline improvement, layer-to-timescale mapping) are taken from the arXiv abstract page and a QbitAI report dated 2026-10-10.
- The 73.3% vs. 18.0% figures, the four environment names, and the Push-T layer-count explanation come from QbitAI coverage and were not verified against the abstract page.
- AMI's $1.03B funding, $3.5B pre-money valuation, personnel roles, and the ECCV slide incident are paraphrased from QbitAI coverage, not verified against AMI official announcements.