PhysiFormer: Learning to Simulate Mechanics in World Space

Forum topic · 小凯 · 2026-06-27

Summary

PhysiFormer is a diffusion transformer for physically-plausible 3D object motion, introduced by Yiming Chen, Yushi Lan, and Andrea Vedaldi (arXiv:2606.27364). Unlike video world models that operate in view-dependent pixel space, PhysiFormer represents objects as 3D meshes expressed in world coordinates. Given the initial vertex positions and velocities, along with the object's material type (rigid or elastic), the model samples future vertex trajectories. Rather than relying on ad-hoc latent spaces or explicitly enforcing rigidity and causality as related neural physics approaches do, PhysiFormer learns physical dynamics directly from data in world space. This makes it a step toward physically grounded world models that can simulate realistic mechanical behavior of 3D objects for computer vision and graphics applications.

Paper Overview

Field: Computer Vision
Authors: Yiming Chen, Yushi Lan, Andrea Vedaldi
Published: 2026-06-27
arXiv: 2606.27364

What Is PhysiFormer?

PhysiFormer is a diffusion transformer designed to generate physically-plausible 3D object motion. Unlike video world models that operate in view-dependent pixel space, PhysiFormer represents objects as 3D meshes expressed in world coordinates.

Given the initial vertex positions and velocities, as well as the object's material type (rigid or elastic), the model samples future vertex trajectories. While related neural physics approaches build on ad-hoc latent spaces or explicitly enforce rigidity and causality, PhysiFormer instead learns mechanics from data directly in world space.

Key Takeaways

World-space representation: 3D meshes in world coordinates instead of pixel space
Diffusion transformer architecture: sampling future vertex trajectories conditioned on initial state and material type
Data-driven physics: learns physical behavior rather than imposing explicit rigidity or causal constraints
Material awareness: supports both rigid and elastic object dynamics

*Automatically collected on 2026-06-27*