HALO: Perceptive Humanoid Motion from Simulation — Synthesized Vision-Language-Motion Data Across a Thousand Procedural Homes

Forum topic · 小凯 · 2026-11-05

Summary

HALO is a humanoid autonomy system from Apple AIML researchers (Siyuan Hu, Vladlen Koltun, et al., arXiv:2611.06908) that addresses the data bottleneck in robot learning. Instead of relying on human teleoperation—which requires an hour of human labor per hour of data—HALO is trained on a pipeline that synthesizes vision-language-motion data across one thousand procedurally generated homes, letting the training corpus scale with compute rather than human effort. The system acquires physical skills including sitting on chairs, navigating to named objects, sidling through half-open doors, crouch-crawling under tables, and pushing boxes, and composes these skills to solve long-horizon tasks. Despite never seeing a real home during training, HALO transfers zero-shot to the physical world, achieving a 91% success rate across 360 trials in six unmapped real environments. Project page: https://apple-aiml-research.github.io/ml-halo/

Paper Overview

Research area: Computer Vision / Robotics Authors: Siyuan Hu, Abulikemu Abuduweili, Yue Meng, Shipeng Liu, Quentin Leboutet, Chen Chen, Matthias Müller, Vladlen Koltun Published: 2026-11-04 arXiv: 2611.06908

Abstract (translated)

Humanoid robots have the potential to automate a wide range of physical tasks. Progress toward generalist humanoid autonomy, however, remains constrained by a data bottleneck: mapping pixels and free-form language to low-level joint commands requires paired supervision, and no corpus of such data exists at sufficiently large scale. Some approaches rely on human teleoperation and wearable capture devices, which require at least an hour of human time to produce an hour of data.

The authors present HALO, a humanoid autonomy system powered by a pipeline that synthesizes vision-language-motion data across a thousand procedurally generated homes. The corpus scales with compute rather than human labor. HALO acquires a repertoire of physical skills, including sitting on chairs, navigating to named objects, sidling through half-open doors, crouch-crawling under tables, and pushing boxes, and coordinates these skills to solve composite long-horizon tasks. Although HALO never saw a real home during training, it transfers zero-shot to the physical world, achieving a 91% success rate over 360 trials in six unmapped real environments.

Code and videos: https://apple-aiml-research.github.io/ml-halo/

---

*Auto-collected on 2026-11-06*

HALO: Perceptive Humanoid Motion from Simulation — Synthesized Vision-Language-Motion Data Across a Thousand Procedural Homes

Summary

Paper Overview

Abstract (translated)

Tags