Inter-X++: A Comprehensive Benchmark for Multimodal Human-Human Interaction

Summary

Researchers propose Inter-X++, a large-scale multimodal benchmark for human-human interaction (HHI) perception and synthesis, essential for building intelligent digital human systems. Existing datasets and models are limited by low-fidelity motion capture, lack of dexterous hand gestures, and scarce multimodal annotations. Inter-X++ was collected with a novel hybrid motion capture system, providing 11,388 high-fidelity interaction sequences with over 8.1 million frames, featuring precise full-body motion and detailed finger articulation. The dataset is enriched with layered fine-grained text descriptions, interaction categories, causal interaction order, subject relationships and personalities, vertex-level contact maps, and physics-regularized constraints. Based on these annotations, the authors establish a unified testbed covering four categories of downstream tasks, symmetrically spanning both generation and perception paradigms. They also introduce OpenHHI, a unified HHI representation and modeling framework that jointly optimizes interaction reconstruction and semantic understanding. Extensive experiments show OpenHHI achieves state-of-the-art performance on both generation and perception tasks. arXiv: https://arxiv.org/abs/2608.20312