MAMHOI: Affordance-Mediated Factorization for Scene-Aware Human-Object Interaction Generation

Forum topic · 小凯 · 2026-10-11

Summary

MAMHOI (arXiv:2610.12416) is a computer vision paper by Mingyuan Lei, Yoonchang Sung, and Tat-Jen Cham addressing scene-aware human-object interaction (HOI) generation in complex 3D scenes. The core challenge is that generating realistic HOI requires both reasoning about interaction feasibility in an environment and synthesizing realistic human-object motion, yet supervision for these two capabilities is rarely available jointly at scale: human-scene datasets offer environment-aware motion, human-object datasets capture interaction dynamics, but paired human-object-scene data is scarce. MAMHOI factorizes generation through an explicit motion-affordance interface: a scene-conditioned model first predicts where and how an interaction can feasibly be performed, then an affordance-conditioned HOI model generates the corresponding human-object motion. This decoupling lets each capability learn from complementary supervision sources without paired data. Experiments in complex indoor environments show MAMHOI reduces object-scene penetration while preserving HOI quality, yielding more realistic, physically plausible interactions. Project page: https://leimingyuan.github.io/MAMHOI-project-page/

Overview

Field: Computer Vision Authors: Mingyuan Lei, Yoonchang Sung, Tat-Jen Cham Published: 2026-10-08 arXiv: 2610.12416

Introduction

Generating realistic human-object interactions (HOI) in complex 3D scenes requires two complementary capabilities: reasoning about interaction feasibility in the environment and synthesizing realistic human-object motion. However, supervision for these capabilities is rarely available jointly at scale. Human-scene datasets provide rich information about environment-aware motion, while human-object datasets capture detailed interaction dynamics — yet paired human-object-scene data remain scarce.

Method

MAMHOI factorizes scene-aware HOI generation through an explicit motion-affordance interface between scene understanding and motion synthesis:

1. Scene-conditioned model: first predicts *where* and *how* an interaction can feasibly be performed in the environment (affordance prediction). 2. Affordance-conditioned HOI model: then generates the corresponding human-object motion.

This factorization allows scene understanding and interaction dynamics to be learned from complementary supervision sources, without requiring paired human-object-scene data.

Results

Experiments in complex indoor environments show that MAMHOI:

Reduces object-scene penetration
Better preserves human-object interaction quality
Produces more realistic, physically plausible scene-aware interactions