Key points
A Tsinghua-led team (with Galbot Inc., Shanghai Jiao Tong University, Peking University, and Shanghai Qi Zhi Institute) presents Humanoid-GPT, a generative pre-training framework for humanoid robots that treats motion control as a language-modeling problem. It achieves zero-shot, real-time tracking of unseen high-dynamic motions (dance, martial arts, sports) on a 29-DoF Unitree G1.
The problem: a 'seesaw' dilemma
- Traditional controllers (AMP, ASE): excel at narrow motion categories but fail on new ones.
- General models (PHC, PMP): cover many motions but master none.
- Root cause: scarce data (millions of frames vs. ~2B tokens for GPT-1) plus architecture bottlenecks.
- Sources: AMASS (~7M frames), LAFAN1 (~2M), Motion-X++ (~20M), MotionMillion, PHUMA, plus in-house captures.
- Cleaning: filter out impossible/irrelevant motions (sitting, swimming, stairs), retarget all skeletons to the 29-DoF Unitree G1 joint space, and apply 5× temporal stretching augmentation.
- Periodic autoencoders extract per-joint periodic amplitudes/frequencies; mean and std of harmonic features form an HME vector.
- K-Means clustering partitions the 2B-frame corpus into ~300 motion clusters (1,000–2,000 sequences each), with no manual motion labels needed.
- Data scaling (2M→2B): Transformer improves near-linearly; MLP and TCN saturate by ~200M frames.
- Model scaling: Transformers improve monotonically; larger MLPs/TCNs overfit on small data.
- First systematic evidence that embodied intelligence follows the same scaling laws as NLP.
- Inference latency: <1.5 ms on a single NVIDIA RTX 4090, 50 Hz control loop, MuJoCo simulation-to-real.
- Training cost: ~15,000 GPU-hours (12,000 on RTX 4090s for PPO experts, 3,000 on H100s for distillation); cluster of 240× RTX 4090 + 24× H100, 3+ days.
- Limited to the 29-DoF G1; new motions require retargeting; proprioception only (no vision); 15k GPU-hours is costly.
- Future: vision-motor fusion, multimodal (voice-to-motion) control, online learning, cross-embodiment transfer.
Data: 2 billion frames of 'robot dialect'
Harmonic Motion Embedding (HME)
Two-stage training: experts then distillation
1. Stage 1: ~384 PPO RL experts, one per motion cluster, trained with keypoint-level rewards (lower-body keypoints weighted 1.5×, upper-body 0.75×; rotation error weight 2.0, position 1.0, velocity 0.03) and aggressive domain randomization (friction 0.3–2.0, terrain up to 0.3 m, random pushes every 5–10 s, mass/CoM perturbations). 2. Stage 2: DAgger distillation of expert trajectories into a single GPT-style causal transformer (SmoothL1 loss, parallel supervision over H=32–64 timesteps).
Model sizes
| Model | Layers | Hidden | Params | Per-step FLOPs | |-------|--------|--------|--------|----------------| | Humanoid-GPT-S | 12 | 192 | 5.38M | 10.98M | | Humanoid-GPT-B | 12 | 384 | 21.37M | 43.18M | | Humanoid-GPT-L | 12 | 768 | 85.21M | 171.30M |
Scaling law validation
Zero-shot results
On AMASS-test, Humanoid-GPT-L reaches 89.35% success rate, MPJPE 0.0732, MPJVE 0.5232, MPKPE 55.15 mm — best or near-best vs. MLP/TCN baselines. Real-world zero-shot demos include dance (Gokuraku Joudo, Michael Jackson-style moonwalk, PokerFace, HuoYuanJia), martial arts (boxing, Chinese kung fu), basketball, single-leg jumps, daily motions (get up, crouch, squat), and collaborative tasks (holding/passing boxes).
Real-robot deployment
Limitations and future work
Conclusion
Humanoid-GPT demonstrates that motion data can be pre-trained like text, that transformer scaling laws hold in the physical world, and that zero-shot generalization is achievable for robots — an 'embodied GPT-1 moment,' per the authors.
Reference: Qi, Z., Chen, X., Wang, J., Lin, C., Lian, Y., Zhang, Z., Zhang, W., Yu, X., Wang, H., & Yi, L. (2026). *Humanoid Generative Pre-Training for Zero-Shot Motion Tracking*. Tsinghua University, Galbot Inc., Shanghai Jiao Tong University, Peking University, Shanghai Qi Zhi Institute.