Deform360: A Massive Multi-view Visuotactile Dataset for Deformable Object World Models

Forum topic · 小凯 · 2026-07-08

Summary

Deform360 is a large-scale visuotactile dataset designed to advance world modeling for deformable object manipulation in robotics. Predicting object dynamics is a fundamental challenge, and deformable objects are especially difficult due to their high-dimensional state spaces and complex material properties. Existing world models learn dynamics in either 2D pixel space or 3D geometric space, but there has been no large-scale real-world dataset to systematically compare their strengths and weaknesses. Deform360 addresses this gap with 198 everyday objects, 1,980 interaction sequences, and over 215 hours of observations, captured by 41 surrounding cameras and a dual-hand tactile gripper that records both global motion and contact-induced local deformation. A markerless visuotactile 3D tracking pipeline extracts dense geometry and motion from the raw data. The authors benchmark current state-of-the-art world models, comparing 2D video diffusion models against 3D particle-based models. Project website: https://deform360.lhy.xyz

Overview

Field: Computer Vision / Robotics
Authors: Hongyu Li, Wanjia Fu, Xiaoyan Cong, Zekun Li, Binghao Huang, Hanxiao Jiang, Xintong He, Yiqing Liang, Rao Fu, Tao Lu, Srinath Sridhar, Kevin A. Smith, George Konidaris, Yunzhu Li
Release date: 2026-07-06
arXiv: 2607.05390
Project website: https://deform360.lhy.xyz

Abstract

World modeling — predicting object dynamics — is a fundamental challenge for robotic manipulation, and deformable objects are particularly hard to model due to their high-dimensional state spaces and complex material properties. Current world models learn dynamics either in 2D pixel space or in 3D geometric space, but the field has lacked large-scale real-world data to systematically understand the relative strengths of each approach.

The Deform360 Dataset

The authors introduce Deform360, a massive multi-view visuotactile dataset featuring:

198 everyday objects
1,980 interaction sequences
215+ hours of observational data
41 surrounding cameras capturing global object motion
Dual-hand tactile grippers capturing contact-induced local deformation

A markerless visuotactile 3D tracking pipeline is used to extract dense geometry and motion from the recordings.

Experiments

Using Deform360, the authors systematically benchmark state-of-the-art world models, comparing 2D video-based models against 3D particle-based models for deformable object dynamics.