ActCam: Zero-Shot Joint Camera and 3D Motion Control for Video Generation

Forum topic · 小凯 · 2026-05-09

Summary

ActCam is a zero-shot video generation method that jointly transfers character motion from a driving video into a new scene while enabling per-frame control of camera intrinsics and extrinsics. Built on any pretrained image-to-video diffusion model conditioned on scene depth and character pose, ActCam generates pose and depth conditions that stay geometrically consistent across frames. It uses a single sampling process with a two-phase conditioning schedule: early denoising steps condition on both pose and sparse depth to reinforce scene structure, after which depth is dropped and pose-only guidance refines high-frequency details to avoid over-constraining generation. Benchmarks spanning diverse character actions and challenging viewpoint changes show ActCam improves camera adherence and action fidelity over pose-only and joint pose-plus-camera baselines, with human evaluators preferring its results, especially under large viewpoint changes. The work demonstrates that careful camera-consistent conditioning and staged guidance achieve strong joint camera and motion control without any training.

Overview

Field: Computer Vision (CV)
Authors: Omar El Khalifi, Thomas Rossi, Oscar Fossey
Published: 2026-05-06
arXiv: 2505.03486

Abstract (translated)

For artistic applications, video generation requires fine-grained control over both performance and cinematography, i.e., the actor's motion and the camera trajectory. The authors present ActCam, a zero-shot video generation method that jointly transfers character motion from a driving video into a new scene and enables per-frame control of intrinsic and extrinsic camera parameters.

ActCam builds on any pretrained image-to-video diffusion model that accepts conditioning in terms of scene depth and character pose. Given a source video with a moving character and a target camera motion, ActCam generates pose and depth conditions that remain geometrically consistent across frames.

A single sampling process with a two-phase conditioning schedule is then used:

1. Early denoising steps condition on both pose and sparse depth to reinforce scene structure. 2. Depth information is then dropped, and pose-only guidance refines high-frequency details, avoiding over-constraining the generation process.

Results

ActCam was evaluated on multiple benchmarks covering diverse character actions and challenging viewpoint changes. Compared with pose-only control and other joint pose-and-camera methods, ActCam improves both camera adherence and action fidelity, and is preferred in human evaluations — especially under large viewpoint changes.

The results indicate that careful camera-consistent conditioning combined with staged guidance enables strong joint camera and motion control without any training.