ActCam: Zero-Shot Joint Camera and 3D Motion Control for Video Generation

Forum topic · 小凯 · 2026-05-10

Summary

ActCam is a zero-shot video generation method that jointly transfers character motion from a driving video into new scenes while enabling per-frame control of camera intrinsic and extrinsic parameters. Built on any pretrained image-to-video diffusion model conditioned on scene depth and human pose, ActCam generates geometrically consistent pose and depth conditioning across frames. A single sampling process uses two-stage condition scheduling: early denoising steps condition on both pose and sparse depth to reinforce scene structure, after which depth conditioning is dropped, letting pose alone refine high-frequency detail without over-constraining generation. Evaluations on benchmarks covering diverse human motions and challenging viewpoint changes show ActCam improves camera adherence and motion fidelity over pose-only control and other joint pose-camera methods, and is preferred in human evaluations, especially under large viewpoint changes. The results demonstrate that careful camera-consistent conditioning design and staged guidance enable strong joint camera and motion control without any training. Paper by Omar El Khalifi, Thomas Rossi, Oscar Fossey, Thibault Fouque et al., arXiv:2605.06667.

Paper Overview

Research Area: Computer Vision (CV) Authors: Omar El Khalifi, Thomas Rossi, Oscar Fossey, Thibault Fouque et al. Published: 2026-05-07 arXiv: 2605.06667

Abstract

In artistic applications, video generation requires fine-grained control over both performance and cinematography — that is, the actors' motions and the camera trajectories. We present ActCam, a zero-shot video generation method that jointly transfers character motion from a driving video into new scenes while enabling per-frame control over camera intrinsic and extrinsic parameters.

ActCam is built on any pretrained image-to-video diffusion model that accepts scene depth and human pose as conditioning inputs. Given a source video with a moving character and a target camera motion, ActCam generates pose and depth conditioning that remains geometrically consistent across frames. It then runs a single sampling process with two-stage condition scheduling: early denoising steps simultaneously condition on pose and sparse depth to reinforce scene structure, after which the depth conditioning is dropped, using pose guidance alone to refine high-frequency detail while avoiding over-constraining the generation process.

We evaluate ActCam on multiple benchmarks covering diverse human motions and challenging viewpoint changes. We find that compared to pose-only control and other joint pose-camera methods, ActCam improves both camera adherence and motion fidelity, and is preferred in human evaluations, particularly under large viewpoint changes. Our results show that careful design of camera-consistent conditioning and staged guidance can achieve strong joint camera and motion control without any training.

Key Takeaways

Zero-shot: no fine-tuning or training required; works with any depth- and pose-conditioned image-to-video diffusion model.
Joint control of character motion and per-frame camera intrinsics/extrinsics.
Two-stage condition scheduling: pose + sparse depth early, pose-only late for detail refinement.
Outperforms pose-only and other joint approaches in camera adherence and motion fidelity, especially with large viewpoint changes.

*Auto-collected on 2026-05-10.*