CamVLA: Calibration-Free View-Robust Vision-Language-Action Models That Self-Locate the Camera

Forum topic · 小凯 · 2026-07-07

Summary

CamVLA (arXiv:2607.05396) is a plug-and-play framework from NTU and Alibaba DAMO Academy that makes vision-language-action (VLA) robot policies robust to camera viewpoint shifts without any calibration. Standard VLAs like π0 collapse when the camera moves: on RLBench, π0's success rate falls from 65.3% to 6.3% under a 15-degree viewpoint offset. CamVLA addresses this by factoring policy into two heads: an Action Head that predicts end-effector deltas in the camera frame (naturally viewpoint-invariant), and a lightweight Geometric Head (6.3M params, 0.19% overhead) that regresses the 6-DoF hand-eye transform from a single RGB image. For delta actions, the predicted translation cancels out mathematically, so only rotation accuracy matters (1.41 degrees error). This lifts π0's average RLBench success from 33.2% to 51.4% on unseen viewpoints and nearly doubles π0's real Franka arm performance at 15-degree offsets, adding only ~1ms inference latency. It even supports a moving handheld camera.

CamVLA: From Fixed to Free Cameras

Paper: From Fixed to Free Cameras: Calibration-Free View-Robust Vision-Language-Action Model Authors: Wenhao Li, Xueying Jiang, Quanhao Qian, Deli Zhao, Shijian Lu, Gongjie Zhang, Ran Xu Affiliations: Nanyang Technological University, Alibaba DAMO Academy, Hupan Lab arXiv: https://arxiv.org/abs/2607.05396

The Problem

State-of-the-art VLA models (π0, GR00T N1, OpenVLA) predict actions in the robot's base frame while observing images in the camera frame. The hand-eye transformation between them is never explicitly modeled — it is implicitly baked into network weights from fixed-view training data. The consequence is catastrophic fragility: on RLBench, π0 drops from 65.3% success at the training viewpoint to 35% at 10 degrees offset and 6.3% at 15 degrees.

Prior view-robust approaches (OC-VLA, 4D-VLA, ray embeddings, AnyCamVLA) all require known camera intrinsics/extrinsics or relative poses — i.e., they need the very calibration information that is unavailable in the deployment scenarios that motivate view robustness in the first place.

Key Points

Factorized architecture: CamVLA adds two parallel heads on top of the VLA visual backbone, inspired by the dorsal/ventral stream separation in human neuroscience.
Action Head: predicts end-effector deltas in the camera frame (ΔA_c = [Δp_c, Δr_c, g]), so visual observations and actions share one coordinate system. A leftward visual translation always corresponds to a negative camera-frame X displacement — a geometrically invariant mapping.
Geometric Head: a tiny 3-layer MLP (1024 hidden, GELU, 6.30M params = 0.19% of the model) regresses the 6-DoF hand-eye pose directly from pooled RGB features. Image-encoder features beat VLM-backbone features for geometry (4.7cm vs 14.7cm translation error) and avoid discontinuous jumps on unseen views.
Elegant math: for delta actions, the hand-eye translation τ cancels exactly — Δp_b = R·Δp_c. So even large translation errors are harmless as long as rotation is accurate; the Geometric Head achieves 1.41° rotation error.

Results

RLBench (training viewpoints sampled every 15°, tested on unseen 5°-spaced views):

| Model | Avg. success | |---|---| | π0 | 33.2% | | π0 + CamVLA | 51.4% | | GR00T N1.7 | 28.4% | | GR00T N1.7 + CamVLA | 38.4% |

Ablations show that simply switching action output to the camera frame accounts for most of the gain (33.2% → ~52%), self-predicted hand-eye loses only 0.9% vs ground truth, and keeping base-frame state input enables true end-to-end calibration-free operation. Predicting only rotation matches full 6-DoF prediction (51.3% vs 51.4%), confirming the translation-cancellation insight.

Real Franka Research 3: at 15° camera offset, π0 falls from 63.3% to 16.0% while CamVLA drops only from 79.0% to 29.3% (nearly double the baseline); at 5° offset CamVLA retains 68.0%. CamVLA also completes tasks with a handheld, continuously moving camera.

Overhead: +0.19% parameters, +0.15% FLOPs, 61ms → 62ms per inference on RTX 4090.

Why It Works

1. Explicit over implicit: the hand-eye transform becomes a network output rather than hidden memorization — more robust, interpretable, and debuggable. 2. Learned geometry over given geometry: no calibration system needed; the model self-locates from a single RGB frame. 3. Factorization: action generation (viewpoint-invariant in camera space) and monocular pose estimation are decoupled into simpler subproblems.

Limitations and Future Work

Monocular third-person camera only; wrist cameras and multi-camera setups are unexplored.
Extreme viewpoint changes and severe occlusions still fail.
Sub-millimeter precision tasks are challenged by small rotation errors.

Natural extensions include wrist/multi-camera support, integration with visual odometry or SLAM, and online hand-eye calibration that cross-checks the Geometric Head's predictions.