Noitom HiPHI, NVIDIA Isaac Video-to-Data, and SONIC: Three Open-Source Pillars for Embodied AI Land Together

Forum topic · QianXun · 2026-08-24

Summary

During the 2026 World Robot Conference (WRC), three complementary open-source releases arrived on the same schedule, forming an industrial-grade reference pipeline for humanoid robotics: data capture, video-to-training-data conversion, and whole-body control. Noitom open-sourced HiPHI, a 617.5-hour (200.1 million frames) sub-millimeter optical motion capture dataset from 132 performers at 90 Hz, split into 371.8 hours of whole-body motion and 245.7 hours of human-object interaction data with synchronized object trajectories and OBJ meshes, organized by FrameNet action semantics for VLA training; models trained on it were deployed on the Unitree G1 for running, sitting, crawling, and carrying tasks. NVIDIA open-sourced Isaac Video-to-Data, a toolchain (ingestion agent, hand/object/body reconstruction, simulation retargeting in Isaac Lab, and CHORD contact-force guidance) that converts human demonstration videos into simulation assets and training data, shipped with a 1,215-task motion library and 121,500 robot episodes. NVIDIA also released SONIC, a 42M-parameter whole-body control model trained on 700 hours of motion capture with a unified token space accepting VR teleoperation, video, text, and music inputs, achieving 99.2% success on 123 real motion sequences on the Unitree G1 and 75% average success on five whole-body VLA tasks. Together they address where high-quality data comes from, how to cheaply convert video into training data, and how to avoid per-skill controllers.

During the 2026 World Robot Conference (WRC), the embodied AI field received three interlocking open-source releases in the same window: motion capture data, video-to-training-data algorithms, and a whole-body control model. Together, they form the first industrial-grade open-source reference implementation of the full loop: where data comes from, how data is used, and how models are trained.

1. Noitom HiPHI: 617.5 hours of sub-millimeter optical motion capture, open-sourced at WRC

Noitom Robotics officially released and open-sourced the HiPHI (High Precision Human Interaction) dataset during WRC, with the data card live on Hugging Face. Three key numbers:

617.5 total hours, 200.1 million frames
132 motion capture performers, 90 Hz capture rate, sub-millimeter optical mocap precision
Split in two: 371.8 hours of whole-body human motion data + 245.7 hours of human-object interaction (HOI) data (including left-right mirroring)

HiPHI differs from prior public human motion datasets in three essential ways:

First, precision tier. This is professional mocap-grade natively captured data, not approximate poses extracted from internet video via pose estimation. Sub-millimeter precision means every joint rotation and finger movement can be mapped back into robot joint space.

Second, synchronized motion + interaction. The HOI portion provides synchronized object trajectories and corresponding OBJ meshes. When a person picks up a cup, sits down, or pushes a door, the robot sees not only the human's motion but the precise 3D position change of the object. This human-object synchronized data is foundational for training robots to understand physical causality.

Third, data organization. Organized by FrameNet action semantics into structured "action units," directly targeting the training needs of VLA (vision-language-action) models.

Commercial use requires separate licensing (officially under the ModalityNet Open Research License, research use only), but for academia and the open-source community this is currently the highest-precision, largest-scale, best-organized open human motion dataset available. Notably, models trained on HiPHI have been deployed on the real Unitree G1, performing running, sitting, crawling, carrying boxes, and dragging luggage—significantly compressing the sim-to-real gap.

2. NVIDIA Isaac Video-to-Data: automatically refining "YouTube videos" into "robot training data"

Where Noitom takes the handcrafted route, NVIDIA takes the mass-refinement route. The same week, NVIDIA open-sourced the Isaac Video-to-Data toolchain, positioned as complementary to HiPHI—it converts real human demonstration videos into simulation assets and training data, then deploys back to physical robots, closing the real→sim→real loop.

The toolchain includes four core modules:

Vision Ingestion Agent: automatically segments long videos and retrieves by action clips
Hand/object/body reconstructors: recovering hand motion, 6-DoF trajectories, depth meshes
Simulation retargeting: retargeting human motion to arbitrary robot bodies, training RL policies in Isaac Lab
CHORD (Contact Wrench Guidance From Human Demonstration): using contact forces to guide dexterous manipulation

Accompanying resources: a motion library of 1,215 tasks, 121,500 robot episodes, and the CHORD training framework, all open-sourced. For small and mid-sized teams that cannot afford mocap equipment or lack real-robot deployment, this is the first toolchain that turns "video" directly into a "training set."

3. SONIC: one model for the whole body—42M parameters trained on 700 hours of mocap

NVIDIA simultaneously released SONIC (Simulating Open-source Neural Intelligence for Control), an open-source whole-body control model. Three numbers frame its capability:

Training data: 700 hours of motion capture, 100M+ motion frames
Model scale: up to 42M parameters
Compute: 128 GPUs, roughly 21,000 GPU hours

SONIC's core innovation is a "universal token space"—different modalities (VR teleoperation, video, text, music) are encoded into shared discrete tokens and fed into a unified robot control decoder. This means one model can simultaneously handle three input types: dancing along to video, generating motion from music, and completing tasks from text instructions.

Real-robot results: 123 real motion sequences on the Unitree G1 with a 99.2% success rate; when connected to a VLA model, an average 75% success rate across five whole-body task categories (reaching + hand-foot coordination). That is industrial-grade sim-to-real transfer capability.

4. What the three releases mean together

For the past three years, humanoid robotics has been stuck in a chicken-and-egg problem: policies trained in simulators don't transfer well to real hardware, and real-robot-capable policies require a separately built controller per skill. The HiPHI + Video-to-Data + SONIC combination answers three questions at once:

Where does high-quality natively captured data come from (HiPHI)
How to convert video data into training data at low cost (Video-to-Data)
Whether whole-body control can work without a per-skill controller (SONIC)

The implication for China's embodied AI track is also clear: during WRC, Noitom chose to open-source core data assets (with commercial licensing reserved), while NVIDIA fully opened its toolchain and model. The complete foundation of "data + toolchain + control model" is being rapidly laid by leading overseas players under a dual "open-source + commercial license" structure. If Chinese embodied AI companies focus only on robot hardware while ceding data, toolchain, and control model ground, they risk being stuck in the awkward position of "other people's training data + other people's control models + our own hardware manufacturing."

Sources: IT Home (2026-08-22, "Noitom Robotics releases HiPHI"), AIQUEST coverage, NVIDIA official GitHub + Hugging Face, Nivlecreto technical analysis, Guangming Online (2026-08-24, "Solving the embodied AI data shortage: inside Beijing Zhongguancun robot training base").