Humanoid-GPT: Applying GPT-Style Pretraining to Humanoid Robot Motion Control

Forum topic · 小凯 · 2026-06-07

Summary

Humanoid-GPT is a Tsinghua-led framework that applies GPT-style causal Transformer pretraining to humanoid whole-body motion control. Trained on 2 billion frames of motion data retargeted to the 29-DoF Unitree-G1 and distilled from approximately 300 reinforcement-learning specialists clustered via Harmonic Motion Embedding (HME), a single transformer achieves sub-1.5 ms inference on an RTX 4090 and tracks unseen dance, martial-arts, and daily motions in zero-shot. Systematic scaling experiments show that Transformers continue improving from 2M to 2B frames and across model sizes, while MLP and TCN baselines saturate early or overfit. The work positions motion control as a learnable language, delivering the first systematic verification of scaling laws in embodied AI and drawing a parallel between Humanoid-GPT and the GPT-1 moment for NLP.

Overview

Humanoid-GPT reframes humanoid whole-body motion control as an autoregressive generation problem. By retargeting 2 billion frames of human motion to a 29-DoF Unitree-G1 joint space, clustering the corpus with Harmonic Motion Embedding, training roughly 300 RL specialists, and distilling them into a GPT-style causal Transformer, the system achieves real-time zero-shot tracking of unseen dance, martial-arts, and everyday actions.

Key Points

Data pipeline: 2B motion tokens retargeted to Unitree-G1, sourced from AMASS, LAFAN1, Motion-X++, MotionMillion, PHUMA, and in-house capture, with 5× temporal stretching and removal of infeasible actions such as sitting or swimming.
Harmonic Motion Embedding (HME): Unsupervised periodic-feature extraction and K-Means clustering yield ~300 motion clusters, removing dependence on action labels.
Two-stage training: ~300 PPO experts, each robust to friction, terrain, payload, and external perturbations, are distilled into a causal Transformer via DAgger with keypoint-level rewards (lower-body weight 1.5×, upper-body 0.75×; rotation α_rot=2.0).
Architecture: Three sizes tested—S (5.38M, 192 dim), B (21.37M, 384 dim), and L (85.21M, 768 dim)—with causal temporal attention and a default 32-frame history extendable to 64.
Scaling-law validation: Transformers scale smoothly from 2M to 2B frames and across model sizes, while MLPs and TCNs saturate at ~200M or overfit on small data, confirming transformer superiority for embodied control.
Zero-shot results (AMASS-test): Humanoid-GPT-L reaches 89.35% success rate, MPJPE 0.0732, MPJVE 0.5232, and MPKPE 55.15 mm, matching or beating MLP/TCN baselines.
Real-world demos: Unseen dances such as *Can Do Can Go!*, *Gokuraku Joudo*, *HuoYuanJia*, *PokerFace*, *Old Town Road*, and Michael-Jackson moonwalk, plus boxing, kung fu, basketball, single-leg jumps, and collaboration tasks, all without fine-tuning.
Deployment: <1.5 ms per forward pass on a single RTX 4090 at 50 Hz in MuJoCo; total training cost 15,000 GPU-hours (12,000 on RTX 4090 for PPO, 3,000 on H100 for distillation).
Limitations: Restricted to 29-DoF topology, requires retargeting for new motions, lacks visual input, and demands substantial compute for academic reproduction.
Outlook: Vision-motion fusion, voice-conditioned generation, online adaptation, and cross-embodiment transfer are identified as the next steps toward a general robot "brain."

Significance

Humanoid-GPT delivers the first systematic evidence that data and model scaling laws from NLP transfer to physical motion. By treating motion as a learnable language and demonstrating stable Transformer scaling where MLP/TCN baselines fail, the authors frame the work as a GPT-1 moment for embodied intelligence.

Reference

Qi, Z., Chen, X., Wang, J., Lin, C., Lian, Y., Zhang, Z., Zhang, W., Yu, X., Wang, H., & Yi, L. (2026). *Humanoid Generative Pre-Training for Zero-Shot Motion Tracking*. Tsinghua University, Galbot Inc., Shanghai Jiao Tong University, Peking University, Shanghai Qi Zhi Institute.