LEXIS: LatEnt ProXimal Interaction Signatures for 3D Human-Object Interaction from a Single Image

Forum topic · 小凯 · 2026-04-24

Summary

LEXIS-Flow is a new diffusion-based framework for reconstructing 3D human-object interaction (HOI) from a single RGB image, introduced by researchers at arXiv:2604.20800. Existing methods rely on sparse binary contact cues, which cannot capture the continuous proximity and dense spatial relationships of natural interactions. The authors first propose InterFields, a representation encoding dense, continuous proximity across the entire body and object surfaces. Since inferring these fields from one image is ill-posed, they exploit the observation that interaction patterns are structured by action and object geometry, learning a discrete manifold of interaction signatures called LEXIS via a VQ-VAE. LEXIS-Flow then uses these signatures to estimate human and object meshes together with their InterFields, whose fields guide refinement for physically plausible, perceptually proximate reconstructions without post-optimization. Evaluations on Open3DHOI and BEHAVE show significant gains over state-of-the-art baselines in reconstruction, contact, and proximity quality, with improved generalization and more realistic results. Code and models are planned for release at https://anticdimi.github.io/lexis.

Paper Overview

Field: Computer Vision
Authors: Dimitrije Antić, Alvaro Budria, George Paschalidis
Published: 2026-04-22
arXiv: 2604.20800
Project page: https://anticdimi.github.io/lexis

Abstract

Reconstructing 3D Human-Object Interaction from an RGB image is essential for perceptive systems. Yet, this remains challenging as it requires capturing the subtle physical coupling between the body and objects. While current methods rely on sparse, binary contact cues, these fail to model the continuous proximity and dense spatial relationships that characterize natural interactions.

We address this limitation via InterFields, a representation that encodes dense, continuous proximity across the entire body and object surfaces. However, inferring these fields from single images is inherently ill-posed. To tackle this, our intuition is that interaction patterns are characteristically structured by the action and object geometry. We capture this structure in LEXIS, a novel discrete manifold of interaction signatures learned via a VQ-VAE.

We then develop LEXIS-Flow, a diffusion framework that leverages LEXIS signatures to estimate the human and object meshes together with their InterFields. Notably, the InterFields help guide refinement, ensuring physically plausible, perceptually proximate reconstructions without post-optimization.

Results

Evaluations on Open3DHOI and BEHAVE show that LEXIS-Flow significantly outperforms existing state-of-the-art baselines in reconstruction, contact, and proximity quality. The approach not only improves generalization but also produces reconstructions judged more realistic, moving closer to holistic 3D scene understanding.

Code and models will be publicly released at https://anticdimi.github.io/lexis.

--- *Auto-collected on 2026-04-24*