CamVLA: A Calibration-Free, View-Robust Vision-Language-Action Model for Robot Manipulation

Forum topic · 小凯 · 2026-07-07

Summary

CamVLA, presented in the arXiv paper 'From Fixed to Free Cameras: Calibration-Free View-Robust Vision-Language-Action Model' by researchers from Nanyang Technological University and Alibaba DAMO Academy, tackles a critical fragility in vision-language-action (VLA) models: shifting a camera by just 15 degrees drops the pi0 model's success rate from 65.3% to 6.3% in RLBench. The method factors the policy into two lightweight heads: an Action Head that predicts end-effector motions in the camera frame (naturally view-invariant), and a Geometric Head that regresses the 6-DoF hand-eye transformation directly from a single RGB image, removing the need for extrinsic calibration. Because delta actions cancel the translation term, only rotation accuracy matters, yielding a rotation error of just 1.41 degrees. CamVLA adds only 0.19% parameters (~6.3M) and 1ms of inference latency, works as a plug-in on pi0 and GR00T N1.7, and lifts average unseen-view success on RLBench from 33.2% to 51.4%. Real-robot tests on a Franka Research 3 under 5-15 degree camera offsets nearly double baseline performance, and the system tracks a handheld moving camera in real time.

CamVLA: Calibration-Free View-Robust Vision-Language-Action Model

Paper: From Fixed to Free Cameras: Calibration-Free View-Robust Vision-Language-Action Model Authors: Wenhao Li, Xueying Jiang, Quanhao Qian, Deli Zhao, Shijian Lu, Gongjie Zhang, Ran Xu Institutions: Nanyang Technological University, Alibaba DAMO Academy, Hupan Laboratory arXiv: https://arxiv.org/abs/2607.05396

The Problem: A 15-Degree Collapse

The paper opens with a striking observation: π0, one of the most advanced vision-language-action (VLA) models, achieves 65.3% success in RLBench under the training viewpoint, but shifting the camera by 15 degrees collapses performance to 6.3% (10° → 35%).

Why it happens

Standard VLAs (OpenVLA, π0, GR00T N1) predict actions in the robot base frame, while observing images in the camera frame. The hand-eye transformation linking the two is never explicitly given or learned—it is implicitly baked into network weights during training with fixed views. The model memorizes visual-motor correspondences rather than understanding geometry, so any camera displacement breaks the mapping.

Weakness of prior work

Existing view-robust approaches all *assume known calibration*:

OC-VLA: actions in a calibrated camera frame (needs extrinsics)
Jiang et al.: ray embeddings (needs intrinsics)
4D-VLA: back-projection using known intrinsics/extrinsics
AnyCamVLA: view synthesis requiring both camera poses

As the authors note: *"Existing view-robust VLAs are therefore most fragile in the deployment regimes that motivate view robustness in the first place."*

Key points

Core idea: factorize "how to move" from "where the camera is," inspired by the dorsal/ventral stream separation in human neuroscience.
Action Head: predicts end-effector deltas in the camera frame (ΔA_c = [Δp_c, Δr_c, g]). Visual-motion alignment becomes geometrically view-invariant: a leftward image translation always maps to a negative displacement along the camera's local X-axis.
Geometric Head: a tiny 3-layer MLP (6.30M params, 0.19% of the model) regresses the 6-DoF hand-eye transform (R, τ) directly from RGB features, achieving 1.41° rotation error under 15° training intervals. Image-encoder features work better than VLM-backbone features (4.7cm vs 14.7cm positional error, no discontinuous jumps).
Elegant math: for delta actions, the translation τ cancels exactly — Δp_b = R·Δp_c — so even large translation errors are harmless as long as rotation R is accurate. Predicting rotation alone matches full 6-DoF prediction (51.3% vs 51.4%).
Truly calibration-free: uses base-frame proprioceptive state input (only −0.3% impact) so no calibration is needed anywhere in the pipeline.

Results

Simulation (RLBench, trained every 15°, tested on unseen every 5° viewpoints)

| Model | Avg. success | |---|---| | π0 | 33.2% | | π0 + CamVLA | 51.4% | | GR00T N1.7 | 28.4% | | GR00T N1.7 + CamVLA | 38.4% |

Ablations show that merely switching the action output to the camera frame accounts for the largest single gain (33.2% → 51.9%); replacing ground-truth hand-eye with self-predicted values costs only ~0.9%.

Real robot (Franka Research 3)

At a 15° camera offset, π0 drops from 63.3% to 16.0%, while π0 + CamVLA reaches 29.3% (from 79.0% nominal). At 5°, CamVLA retains 68.0% vs π0's 53.3%. The system even tracks a handheld moving camera in real time, going beyond any one-time calibration paradigm.

Overhead

Only +0.19% parameters (3238.1M → 3244.4M), +0.15% FLOPs, and +1 ms inference latency (61→62 ms on RTX 4090) — negligible for 10 Hz control.

Limitations

Supports only a monocular third-person camera; wrist cameras and multi-camera fusion are unexplored.
Extreme viewpoint changes (object out of view, heavy occlusion) still fail.
Sub-millimeter-precision tasks are challenged by amplified rotational errors.

Natural extensions include integrating visual odometry/SLAM for large displacements and online hand-eye self-correction.

Takeaway

CamVLA demonstrates that view robustness can be achieved not by bigger models or more data, but by a geometric restructuring of the problem: express actions in the observation frame, and infer the camera pose explicitly rather than memorizing it implicitly. With only 6.30M added parameters, it resolves a core deployment bottleneck of VLA models—a reminder that classical geometry and deep learning are a powerful combination.

References

1. Li et al. (2026). From Fixed to Free Cameras: Calibration-Free View-Robust Vision-Language-Action Model. arXiv:2607.05396 2. Black et al. (2024). π0: A vision-language-action flow model. RSS 3. Bjorck et al. (2025). GR00T N1. arXiv:2503.14734 4. Kim et al. (2024). OpenVLA. CoRL 5. James et al. (2020). RLBench. IEEE RA-L 5(2):3019-3026 6. Goodale & Milner (1992). Separate visual pathways for perception and action. Trends in Neurosciences 15(1):20-25