OmniRobotHome: A Multi-Camera Platform for Real-Time Multiadic Human-Robot Collaboration in Home Environments

Forum topic · 小凯 · 2026-05-02

Summary

OmniRobotHome (arXiv:2604.28197) by Naoki Kita, William J. Beksi, and Luis Sentis is the first room-scale residential research platform that unifies wide-area, real-time 3D human and object perception with coordinated multi-robot execution in a single world coordinate frame. The system deploys 48 hardware-synchronized RGB cameras in a natural home setting, enabling markerless, occlusion-robust tracking of multiple humans and objects, time-aligned with the real-time scene-state execution of two Franka robot arms. This addresses the central bottleneck of multiadic human-robot collaboration: close-proximity interactions among humans, robots, and objects cause persistent occlusion and rapid state changes. Continuous capture in a consistent coordinate frame also enables long-horizon human behavior modeling from accumulated trajectories. The authors demonstrate measurable improvements on two core problems: safety in shared human-robot workspaces and human-anticipatory robot assistance.

Paper Overview

Field: Robotics / Computer Vision (RO/CV)
Authors: Naoki Kita, William J. Beksi, Luis Sentis
Published: 2026-04-30
arXiv: 2604.28197

Abstract (English)

Human-robot collaboration has been studied primarily in dyadic or sequential settings. However, real homes require multiadic collaboration, where multiple humans and robots share a workspace, acting concurrently on interleaved subtasks with tight spatial and temporal coupling. This regime remains underexplored because close-proximity interaction between humans, robots, and objects creates persistent occlusion and rapid state changes, making reliable real-time 3D tracking the central bottleneck.

Key Points

First room-scale home platform: OmniRobotHome integrates wide-area real-time 3D human and object perception with coordinated multi-robot execution in a unified world coordinate frame.
48 hardware-synchronized RGB cameras are deployed in a natural home environment, enabling markerless, occlusion-robust tracking of multiple humans and objects.
Robot integration: Perception is temporally aligned with the real-time scene-state execution of two Franka robot arms.
Long-horizon behavior modeling: Continuous capture in a consistent coordinate frame supports modeling of human behavior from accumulated trajectories.
Demonstrated benefits: The authors show that real-time perception and accumulated behavioral memory yield measurable improvements in (1) safety in shared human-robot environments and (2) human-anticipatory robot assistance.

The platform makes multiadic collaboration scenarios experimentally feasible, opening a previously underexplored research direction in human-robot interaction.

--- *Auto-collected on 2026-05-02*