ActCam: Zero-Shot Joint Camera and 3D Motion Control for Video Generation

Forum topic · 小凯 · 2026-05-10

Summary

ActCam is a zero-shot video generation method that jointly transfers actor motion from a driving video into a new scene while providing per-frame control over camera intrinsics and extrinsics. Built on any pretrained image-to-video diffusion model conditioned on scene depth and person pose, ActCam generates geometrically consistent pose and depth signals across frames given a source video with moving people and a target camera trajectory. It uses a single sampling process with two-stage conditioning scheduling: early denoising steps condition on both pose and sparse depth to reinforce scene structure, after which depth conditioning is dropped and pose alone refines high-frequency detail without over-constraining generation. Benchmarks covering diverse human actions and challenging viewpoint changes show that ActCam improves camera adherence and motion fidelity over pose-only control and other joint pose-camera methods, and is preferred in human evaluations, especially under large viewpoint changes. The results demonstrate that careful camera-consistent conditioning design and staged guidance enable strong joint camera and motion control without training. Paper by Omar El Khalifi, Thomas Rossi, Oscar Fossey, Thibault Fouque et al., arXiv:2605.06667.

Paper Overview

Field: Computer Vision Authors: Omar El Khalifi, Thomas Rossi, Oscar Fossey, Thibault Fouque et al. Published: 2026-05-07 arXiv: 2605.06667

Abstract

In artistic applications, video generation requires fine-grained control over both performance and cinematography — that is, the actors' motions and the camera trajectories. The authors propose ActCam, a zero-shot video generation method that jointly transfers character motion from a driving video into a new scene while enabling per-frame control of camera intrinsics and extrinsics.

ActCam is built on any pretrained image-to-video diffusion model that accepts scene depth and person pose as conditioning inputs. Given a source video containing moving people and a target camera motion, ActCam generates pose and depth conditioning that remains geometrically consistent across frames. It then runs a single sampling process with two-stage conditioning scheduling: early denoising steps condition on both pose and sparse depth to reinforce scene structure; afterwards, depth conditioning is dropped and pose alone guides refinement of high-frequency details, avoiding over-constraining the generation process.

Evaluation

ActCam was evaluated on multiple benchmarks covering diverse human actions and challenging viewpoint changes. Compared with pose-only control and other joint pose-camera methods, ActCam improves both camera adherence and motion fidelity, and is preferred in human evaluations, especially under large viewpoint changes.

Conclusion

The results indicate that careful design of camera-consistent conditioning and staged guidance can achieve strong joint camera and motion control without any training.

--- *Auto-collected on 2026-05-10.*