CalTennis: Large-Scale Multi-View Tennis Video Dataset and Monocular-to-3D Pose Estimation Benchmark

Forum topic · 小凯 · 2026-06-23

Summary

CalTennis is a large-scale video benchmark for evaluating monocular-to-3D human pose estimation in the wild, introduced by researchers at Caltech (arXiv:2506.18490). The dataset contains over 11 million frames (51 hours) of tennis practice and match play from 40 players, captured with 2-6 synchronized cameras at 60 Hz. It is 10x larger than existing in-the-wild human motion video datasets, 3x larger than existing MOCAP-ground-truthed datasets, and the first large-scale benchmark providing synchronized multi-view recordings of expert athletic motion. The multi-view setup enables inexpensive, label-free evaluation of monocular-to-3D pose algorithms, supported by a simple standardized collection protocol requiring no specialized equipment, plus fully automated video calibration and synchronization. Benchmarking state-of-the-art monocular-to-3D pose methods on CalTennis shows that 3D joint angle recovery is now reasonably accurate, while all models still struggle with depth estimation and foot-ground contact consistency. The authors also propose two novel metrics—footwork and stability—and a qualitative study of body shape inconsistencies, revealing underexplored failure modes and concrete improvement opportunities in pose estimation and motion analysis.

Paper Overview

Field: Computer Vision
Authors: Ilona Demler, Xinran Xie, Blake Werner
Published: 2025-06-23
arXiv: 2506.18490

Summary

The Caltech Tennis Dataset (CalTennis) is a large-scale video benchmark for evaluating monocular-to-3D pose estimation in the wild. CalTennis comprises over 11 million frames (51 hours) of tennis practice and match play from 40 players, captured with 2-6 synchronized cameras at 60 Hz.

It is 10 times larger than existing in-the-wild human motion video datasets and 3 times larger than existing MOCAP-ground-truthed datasets, and it is the first large-scale benchmark to provide synchronized multi-view recordings of expert athletic motion.

Key Points

Scale: 11M+ frames (51 hours), 40 players, tennis practice and match footage.
Multi-view capture: 2-6 synchronized cameras at 60 Hz, enabling inexpensive, label-free evaluation of monocular-to-3D pose estimation algorithms.
Simple protocol: A standardized data collection workflow requiring no specialized equipment or expertise, with fully automated video calibration and synchronization.
Benchmark findings: State-of-the-art monocular-to-3D pose methods achieve fairly accurate 3D joint angle recovery, but all models still struggle with depth estimation and foot-ground contact consistency.
New metrics: The paper introduces two novel performance measures—footwork and stability—plus a qualitative study of body shape inconsistencies.
Implications: These metrics reveal previously underexplored failure modes and point to concrete improvement opportunities in pose estimation and motion analysis.

Original Abstract (excerpt)

> The Caltech Tennis Dataset (CalTennis) is a large-scale video benchmark for evaluating monocular-to-3D pose estimation in the wild. CalTennis comprises over 11 million frames (51 hours) of tennis practice and match play from 40 players, captured with 2-6 synchronized cameras at 60 Hz. It is 10 times larger than existing in-the-wild human motion video datasets and 3 times larger than existing MOCAP-ground-truthed datasets...

---

*Auto-collected on 2026-06-23*

CalTennis: Large-Scale Multi-View Tennis Video Dataset and Monocular-to-3D Pose Estimation Benchmark

Summary

Paper Overview

Summary

Key Points

Original Abstract (excerpt)

Tags