TryOnCrafter: Camera-Controllable Video Virtual Try-on via a Renderable 4D Try-on Proxy

Forum topic · 小凯 · 2026-06-26

Summary

TryOnCrafter is a new framework for camera-controllable video virtual try-on (CaM-VVT), addressing a key limitation of existing video virtual try-on (VVT) systems: their passive dependence on source camera trajectories, which prevents free omnidirectional viewpoint exploration. Introduced by Hao Sun, Hao Yan, and Mengting Chen (arXiv:2606.19229), TryOnCrafter is the first unified DiT-based framework designed for the CaM-VVT task. Instead of implicit pixel-space manipulation, it introduces a renderable 4D try-on proxy that explicitly decouples the human subject from the environment. High-fidelity 2D try-on priors are distilled into a 3DGS-based clothed avatar, animated via SMPL-X sequences and metrically aligned with a reconstructed background point cloud. This proxy provides a robust structural foundation with dense textures and motion completeness, serving as a geometric anchor for the video DiT so that generated videos strictly follow prescribed camera trajectories while maintaining physically plausible deformations. The proxy's editability also enables downstream applications such as human repositioning, bullet-time effects, and 360-degree orbit viewing.

Overview

Field: Computer Vision Authors: Hao Sun, Hao Yan, Mengting Chen Published: 2026-06-25 arXiv: 2606.19229

Abstract

While Video Virtual Try-on (VVT) has achieved remarkable progress in synthesizing realistic garment overlays on dynamic subjects, existing paradigms remain fundamentally constrained by a passive dependency on source camera trajectories, failing to accommodate the requisite interactive freedom for omnidirectional viewpoint exploration. To address this limitation, the authors define a pioneering research frontier: Camera-controllable Video Virtual Try-on (CaM-VVT).

Unlike conventional VVT, CaM-VVT not only necessitates viewpoint-agnostic texture hallucination but also strict structural synchronization between non-rigid human dynamics and background contexts under arbitrary, unconstrained camera movements. To tackle these challenges, the authors present TryOnCrafter, the first unified DiT-based framework specifically built for the CaM-VVT task.

Key Contributions

Renderable 4D try-on proxy: Instead of implicit pixel-space manipulation, TryOnCrafter explicitly decouples the human subject from the environment.
3DGS-based clothed avatar: High-fidelity 2D try-on priors are distilled into a 3D Gaussian Splatting avatar, animated via SMPL-X sequences and metrically aligned to a reconstructed background point cloud. This proxy establishes a robust structural foundation with superior texture density and motion completeness.
Geometry-anchored video DiT: The video DiT leverages the proxy as the primary geometric anchor, ensuring that synthesized photorealistic videos strictly follow the prescribed camera trajectory and exhibit physically plausible deformations.
Editable downstream applications: Thanks to the inherent editability of the 4D proxy, TryOnCrafter enables diverse applications, including human repositioning, "bullet-time" effects, and 360-degree orbit viewing.