VioLA: Learning Generalist Humanoid Control Policies from Human Data

Forum topic · 小凯 · 2026-10-11

Summary

VioLA is a generalist humanoid robot control policy that addresses two key obstacles in whole-body humanoid control: the large, tightly coupled joint-level action space, and the scarcity of robot demonstration data. Instead of predicting joint commands, VioLA predicts body and hand motion latents that are executed by pretrained low-level controllers. Motion encoders map both human and robot motion into the same latent space, so human videos serve as natural supervision. Its training pool contains 140.6 million frames, 93.2% of which come from human data. On real hardware, VioLA follows locomotion instructions zero-shot with a 100% success rate (compared to 16.7% for GR00T N1.7 and 0% for Psi-0) and achieves 88.6% success on manipulation tasks, all without task-specific fine-tuning. The method works across two VLA backbones and one world-action model. A generalist policy trained purely on human demos can zero-shot perform locomotion tasks on real hardware. Code and checkpoints will be released (arXiv:2610.12435).

Paper Overview

Field: Machine Learning Authors: Mert Albaba, Jens Beißwenger, Anna Manasyan, Daniel Marta, Michael J. Black, Wieland Brendel, Andreas Krause, Georg Martius, Martin Riedmiller Posted: 2026-10-08 arXiv: 2610.12435

What Problem Does VioLA Solve?

Teaching a humanoid robot to follow instructions with its whole body faces two obstacles:

1. Large, tightly coupled action space — legs, arms, and fingers must move together while the robot keeps its balance, making joint-level actions hard to learn. 2. Scarce humanoid demonstrations — current humanoid generalist policies do not follow new instructions out of the box and require fine-tuning on teleoperated demonstrations of each task before deployment.

Human demonstrations exist in far larger numbers, but a person's motion is not a robot command.

Approach

VioLA removes both obstacles by changing what the generalist policy predicts:

Instead of joint commands, VioLA predicts body and hand motion latents.
A pretrained body- and hand-controller executes these latents on the robot.
Motion encoders map both human and robot motion into the same latent space, so a human video is naturally annotated in the policy's action space.
The training demonstration pool contains 140.6 million frames, 93.2% of which come from human data.

Results

Zero-shot locomotion on real hardware: 100% success rate, without task-specific fine-tuning (GR00T N1.7: 16.7%; Ψ₀: 0%).
Manipulation tasks: 88.6% success rate, also without task-specific fine-tuning.
The method generalizes across two VLA backbones and one world-action model backbone.
A generalist policy trained only on human demonstrations can zero-shot execute locomotion tasks on real hardware.

Code and model checkpoints will be publicly released.

Full Abstract

> Teaching a humanoid to follow instructions with its whole body runs into two obstacles. Its action space is large and tightly coupled: legs, arms, and fingers must move together while the robot keeps its balance, which makes joint-level actions hard to learn. And humanoid demonstrations are scarce, so current humanoid generalist policies do not follow new instructions out of the box and are fine-tuned on teleoperated demonstrations of each task before deployment. Human demonstrations exist in far larger numbers, but a person's motion is not a robot command. We remove both obstacles by changing what the generalist policy predicts. We introduce VioLA, a generalist humanoid policy that predicts body and hand motion latents instead of joint commands. A pretrained body- and hand-controller exec...

---

*Auto-collected on 2026-10-11*