ActCam: Zero-Shot Joint Camera and 3D Motion Control for Video Generation

Forum topic · 小凯 · 2026-05-10

Summary

ActCam is a zero-shot video generation method that jointly transfers actor motion from a driving video into a new scene while providing per-frame control over both intrinsic and extrinsic camera parameters. Built on any pretrained image-to-video diffusion model conditioned on scene depth and human pose, ActCam first generates geometrically consistent pose and depth conditions across frames given a source video with a moving person and a target camera trajectory. It then runs a single sampling process with a two-stage condition schedule: early denoising steps are conditioned jointly on pose and sparse depth to enforce scene structure, after which the depth conditioning is dropped so that only pose guidance refines high-frequency details without over-constraining generation. Evaluations on benchmarks covering diverse human motions and challenging viewpoint changes show that ActCam improves camera adherence and motion fidelity compared with pose-only control and other joint pose-camera methods, and is preferred in human evaluations, especially under large viewpoint changes. The results demonstrate that careful camera-consistent conditioning and staged guidance can achieve strong joint camera-motion control without any training. Paper: arXiv 2605.06667.

Introduction

This post summarizes the paper ActCam: Zero-Shot Joint Camera and 3D Motion Control for Video Generation.

Research area: Computer Vision (CV)
Authors: Omar El Khalifi, Thomas Rossi, Oscar Fossey, Thibault Fouque, et al.
arXiv: 2605.06667

Key Points

In artistic applications, video generation requires fine control over both performance and cinematography — that is, the actor's motion and the camera trajectory. The authors propose ActCam, a zero-shot video generation method that jointly transfers the actor's motion from a driving video into a new scene while enabling per-frame control of both intrinsic and extrinsic camera parameters.

Method

ActCam is built on any pretrained image-to-video diffusion model that accepts scene depth and human pose as conditions.
Given a source video containing a moving person and a target camera motion, ActCam generates geometrically consistent pose and depth conditions across frames.
A single sampling process is run with a two-stage condition schedule:
Early denoising steps are conditioned jointly on pose and sparse depth to reinforce scene structure.
The depth conditioning is then dropped; only pose guidance refines high-frequency details, avoiding over-constraining the generation process.

Results

Evaluated on multiple benchmarks covering diverse human motions and challenging viewpoint changes.
Compared with pose-only control and other joint pose–camera methods, ActCam improves camera adherence and motion fidelity.
ActCam is preferred in human evaluations, especially under large viewpoint changes.
The results show that careful camera-consistent conditioning design and staged guidance enable strong joint camera and motion control without any training.

Reference

Paper: arXiv:2605.06667

---

*Auto-collected on 2026-05-10*