How many numbers does a humanoid robot emit per second? In Holo-M's setup, the answer is 96 dimensions × 30 steps — 2,880 floats. Legs move, torso moves, and each hand has a dozen-plus joints moving, all with different scales and frequencies.
Manipulator VLA models have already proven discrete action tokens work. The humanoid problem is that whole-body action space is far higher-dimensional and heterogeneous. Holo-M (arXiv:2609.35709) slices those 2,880 numbers into four groups, tokenizes each separately, then lets the language model emit them all in one shared vocabulary.
Four tokenizers, one 96-dim action space
Holo-M splits the action space by body part into four groups: end-effectors, body, hands, and kinematics. Each group is first compressed along the time axis with a discrete cosine transform, keeping only the top 2 low-frequency coefficients per action dimension; the residual is discretized with 4-level residual vector quantization (RVQ). Codebooks are fit with k-means — no extra neural networks trained.
| Group | Content | Raw dims | Tokens | |---|---|---|---| | End-effectors | dual wrist poses + fingertip positions | 48 | 100 | | Body | torso joint targets | 29 | 62 | | Hands | hand joint targets | 14 | 32 | | Kinematics | base velocity, height, yaw rate | 5 | 14 | | Total | one 30-step action chunk | 96 | 208 |
The end-effector group is a shared task-space representation, letting three very different data sources share one vocabulary in training: humanoid teleoperation data, egocentric human video, and simulation data. Missing groups aren't imputed — a validity mask simply excludes them from the loss. An EgoDex sample with only end-effectors valid contributes only its 100 tokens.
The backbone is Qwen3-VL-2B with a frozen vision encoder; only the language backbone is optimized. The vocabulary is extended with state, action, and structural tokens. Action tokens live in the same vocabulary as language tokens, generated by the same sequence model — no separate continuous action expert, no diffusion head.
From 208 sequential steps to 32
Autoregressively decoding a chunk takes 208 sequential forward passes — too expensive for a 30 Hz control loop.
Holo-M uses grouped discrete diffusion decoding: each of the four groups forms a diffusion block, ordered end-effectors → kinematics; tokens within a block interact bidirectionally and reconstruct in parallel, while blocks stay causal with respect to each other. At inference, the model predicts all masked positions each step and unmasks by confidence, appending a finished block to context before decoding the next.
With 8 unmasking steps, the four blocks take 32 sequential decoding steps total — 6.5× fewer than 208 autoregressive steps.
Latency was measured on a single RTX 5090; the robot receives commands over WebSocket.
| Unmasking steps | Hold budget | Inference latency (ms) | |---|---|---| | 2 | 233 | 175.8 ± 11.5 | | 4 | 333 | 275.7 ± 9.3 | | 8 | 500 | 476.8 ± 16.9 |
All three fit within budget. Simulation evals use 8 steps (a 500 ms observation period). The control loop is fixed-rate: new observations every 15 ticks; while the model infers a new chunk, the robot keeps executing the previous one; timed-out chunks are dropped and the robot returns to its last pose.
Two sets of numbers on SIMPLE
The SIMPLE benchmark uses 6 representative tasks × 3 domain-randomization levels × 10 trials = 180 points max.
Generalist mode (one checkpoint jointly post-trained on all 24 tasks, no per-task adaptation):
| Method | Total score | |---|---| | Holo-M | 143 / 180 | | Holo-M autoregressive | 138 / 180 | | Ψ0 | 114 / 180 | | ACT | 79 / 180 | | DreamZero | 38 / 180 | | π0.5 | 28 / 180 |
Specialist mode (fine-tuned per task from the same generalist checkpoint): Holo-M 163/180, Holo-M autoregressive 157/180, Ψ0 154/180, ACT 149/180, DreamZero 118/180. The largest gap is on long-horizon mobile pick-and-place: Holo-M 25/30 vs. Ψ0's 18/30.
Notably, the autoregressive variant alone already beats all baselines — meaning the main gains come from unified tokenization and joint training, with grouped diffusion adding a further increment: slightly higher success rates and a decoding depth drop from 208 to 32.
Pretraining data matters just as much: no pretraining yields 53/180; adding Humanoid Everyday humanoid data doubles it to 119/180; adding EgoDex egocentric human video brings it to 138/180 — 2.6× total.
Only two real-robot tasks so far
The deployment platform is a Unitree G1-comp competition unit with two Dex3 dexterous hands and a top-mounted RealSense camera. Inputs: 640×360 30 Hz RGB from the head camera, 31 upper-body joint angles (each discretized to 256 state tokens), and language instructions. No lower-body joints, base velocity, height, or IMU.
Real-robot success: tabletop grasping 10/10, mobile pick-up 8/10, over 10 rollouts each. The specialist checkpoints were fine-tuned with 100 teleoperated episodes per task.
What to watch
The design cleanly decomposes a hard problem: heterogeneous data sources coexist via group masking, real-time decoding via intra-block parallelism and inter-block causality, all without extra model parameters.
The caveat is validation scope. All SIMPLE scores come from simulation; real-robot results cover only two tasks at 10 trials each, neither strongly coupling lower body and base. The paper decodes the kinematics group last, meaning base commands depend on the outputs of the first three groups — that dependency's stability in long-horizon navigation needs more real-robot data to judge.
The team says all code and model weights will be released. Once that happens, whether this 208-token scheme becomes a universal interface for humanoid whole-body control will get third-party answers quickly.