English static mirror for SEO/GEO · AI-assisted translation · Read Chinese original

Cambrian-P: Pose-Grounded Video Understanding

Forum topic · 小凯 · 2026-05-25

Summary

Cambrian-P is a video multimodal large language model (MLLM) that incorporates camera pose as a lightweight supervision signal for video understanding. The authors argue that pose—defining the position and orientation of each viewpoint within a shared spatial coordinate frame—is largely missing from current MLLMs, which treat video frames as isolated 2D snapshots. Cambrian-P adds per-frame learnable camera tokens and a pose regression head to a video MLLM. With a carefully designed sampling scheme, the model achieves 4.5–6.5% gains on spatial reasoning benchmarks such as VSI-Bench, generalizes to eight additional spatial and general video QA benchmarks, and, as a byproduct, attains state-of-the-art streaming pose estimation on ScanNet. Notably, training on pseudo-labeled poses from in-the-wild videos further improves general video QA, suggesting pose benefits extend beyond spatial reasoning. The paper is available at arXiv:2505.14485.

Paper Overview

Field: Computer Vision (CV) Authors: Jihan Yang, Zifan Zhao, Xichen Pan Published: 2026-05-25 arXiv: 2505.14485

Abstract

Camera pose matters. The position and orientation of each viewpoint define a shared spatial coordinate frame that links observations across video frames. Yet this signal is largely absent from multimodal LLMs (MLLMs) used for video understanding, which treat frames as isolated 2D snapshots rather than a continuous scene as humans perceive it.

The authors revisit pose as a lightweight supervision signal and introduce Cambrian-P, an enhanced video MLLM equipped with per-frame learnable camera tokens and a pose regression head. Through a carefully designed sampling scheme, the model achieves:

  • Significant gains of 4.5–6.5% on spatial reasoning benchmarks such as VSI-Bench
  • Generalization to eight additional spatial and general video QA benchmarks
  • As a byproduct, state-of-the-art streaming pose estimation on ScanNet
Surprisingly, training on pseudo-labeled poses from in-the-wild videos further improves general video QA benchmarks, indicating that pose plays a role beyond spatial reasoning. Together, these results position camera pose as a fundamental signal for video models that reason about the physical world.

---

*Auto-collected on 2026-05-25*

Tags

#cambrian-p#video-understanding#camera-pose#mllm#spatial-reasoning#computer-vision#arxiv

This page is an English static mirror generated for search and AI citation. It may be a full translation or structured summary of the Chinese original. Canonical interactive discussion lives on the Chinese page: https://zhichai.net/topic/177620758