ARDY: Autoregressive Diffusion with Hybrid Representation for Interactive Real-Time 3D Human Motion Generation

Forum topic · 小凯 · 2026-07-13

Summary

ARDY is a streaming motion generation framework from NVIDIA Research (arXiv:2507.08713, July 2025) that generates realistic 3D human motions in real time for interactive applications such as animation, simulation, and humanoid robotics. It addresses a key trade-off: offline text- and constraint-conditioned methods offer precise control but are too slow for interactive use, while online real-time methods sacrifice controllability or struggle with complex text semantics and long-horizon goals due to limited context windows. ARDY uses a hybrid representation combining explicit root features with latent body embeddings, balancing precise trajectory control and efficient generative learning. A two-stage autoregressive transformer denoiser with variable historical context conditions on flexible, long-horizon kinematic constraints. Trained on large-scale motion capture data with text labels and kinematic constraints sampled from ground-truth poses, ARDY supports online prompting and long-term goals. Evaluations on HumanML3D and the high-fidelity Bones Rigplay dataset show strong motion quality and constraint adherence. An interactive demo demonstrates dynamic text control, keyframe pose constraints, path following, and mouse/keyboard motion control.

Overview

Field: Computer Vision (motion generation)
Authors: Kaifeng Zhao, Mathis Petrovich, Haotian Zhang, et al.
Published: 2025-07-12
arXiv: 2507.08713
Project page: https://research.nvidia.com/labs/sil/projects/ardy/

Problem

Generating realistic 3D human motions in real time within interactive applications is key for animation, simulation, and humanoid robotics. Existing approaches face a trade-off:

Offline methods offer precise control via text and kinematic constraints, but lack the inference speed required for interactive settings.
Online methods achieve real-time synthesis but often sacrifice controllability, or struggle with complex text semantics and long-horizon goals due to limited context windows.

Method

ARDY is a streaming generation framework that bridges this gap by enabling high-fidelity motion generation controllable via online text prompts and flexible kinematic constraints.

Hybrid representation: combines explicit root features with latent body embeddings, balancing precise trajectory control with efficient generative learning.
Two-stage autoregressive transformer denoiser: uses variable historical context and supports conditioning on flexible, long-horizon kinematic constraints.
Training: on large-scale motion capture datasets, conditioned directly on text labels and kinematic constraints sampled from ground-truth poses, so ARDY naturally learns controllable generation supporting online prompting and flexible long-term goals.

Results

Extensive evaluation on the HumanML3D benchmark and the large-scale, high-fidelity Bones Rigplay dataset demonstrates high motion quality and strong constraint adherence, validating the key architectural decisions.
An interactive demo showcases practical versatility: dynamic text control, diverse keyframe pose constraints, path following, and interactive motion control via mouse and keyboard.

Supplementary videos, code, and models are available at https://research.nvidia.com/labs/sil/projects/ardy/.