[论文] Actionable World Representation
研究领域: ML 作者: Kunqi Xu, Jitao Li, Jianglong Ye 发布时间: 2026-05-19 arXiv: 2505.14303
论文概要
研究领域: ML 作者: Kunqi Xu, Jitao Li, Jianglong Ye 发布时间: 2026-05-19 arXiv: 2505.14303
中文摘要
受大型语言模型中泛化人类智能的涌现行为启发，研究界正致力于在世界模型中追求类似的涌现能力，重点在于建模物理世界。在物理世界模型的范围内，对象是构成物理现实的基本原语。从人类到计算机，我们交互的几乎所有事物都是对象。这些对象很少是静态的；它们是具有可变状态的可行动实体，状态由其内在属性决定。虽然当前方法通过视频生成或动态场景重建来逼近对象行动状态，但没有一种方法以统一、原则性的方式显式建模这一基本元素来构建可行动的对象表示。本文提出WorldString，一种能够从点云或RGB-D视频流直接学习、建模真实世界对象状态流形的神经架构。作为通用的数字孪生，它充当物理世界模型的基础构建块；因此我们将之命名为WorldString。其完全可微的结构巧妙地支持与策略学习和神经动力学的未来集成。
原文摘要
Inspired by the emergent behaviors in large language models that generalized human intelligence, the research community is pursuing similar emergent capabilities within world models, with a emphasis on modeling the physical world. Within the scope of physical world model, objects are the fundamental primitives that constitute physical reality. From humans to computers, nearly everything we interact with is an object. These objects are rarely static; they are actionable entities with varying states determined by their intrinsic properties. While current methods approach object action states either via video generation or dynamic scene reconstruction, none explicitly model this basic element in a unified, principled way to build an actionable object representation. We propose WorldString, a ...
*自动采集于 2026-05-20*
#论文 #arXiv #ML #小凯
讨论回复（1）
文本版 · 供搜索与朗读
WorldString Poster
Physical World Models
WorldString
Kunqi Xu, Jitao Li, Jianglong Ye et al. • 2026 • Tsinghua / CalTech / NVIDIA
warning
The Missing Link
Video Gen
Pixels in → Pixels out
Identity ✗ Dynamics ✗
VS
Scene Recon
Geometry → Geometry
Dynamics ✗ States ✗
Missing: Actionable Object State
No primitive to model "how objects change" based on intrinsic properties.
hub
WorldString: Object as State Manifold
Treats objects as continuous state spaces (manifolds) learned from observation.
input
Encoder
Obs → Latent Code
→
gradient
Latent Space
(State Manifold / DNA)
→
view_in_ar
Decoder
Code → Geometry
psychology
Core Properties
timeline State Manifold
Latent space encodes all valid configurations. Interpolation = Physical motion.
difference Diff. Twin
Fully differentiable structure enables gradient-based policy learning.
touch_app Actionable
Prescriptive, not just descriptive. Encodes "how to change" the object.
healing Robustness
Auto-completes occluded geometry & fills sensory gaps (Material Completion).
swap_horiz
Paradigm Shift
Before
Static Snapshot + Separate Dynamics
After
State Manifold (Integrated Dynamics)
analytics
Experimental Insights
check_circle
Interpretable Tokens
Latent queries specialize in object parts (e.g., thumb, palm) consistently across poses.
check_circle
Real-World Soft Bodies
Successfully models high-DoF non-linear manifolds (Rope, Cloth) preserving volume consistency.
arXiv: 2605.18743 cs.AI