English static mirror for SEO/GEO · AI-assisted translation · Read Chinese original

HumanCLAW: Can Vision-Language Models Act Through a Body?

Forum topic · 小凯 · 2026-07-31

Summary

HumanCLAW is an evaluation framework that decouples action decision-making from low-level motor execution to test whether vision-language models (VLMs) can act through a physical body. At each step, a harness wraps an off-the-shelf VLM, which issues atomic skill commands that are translated into sub-second chunks of continuous full-body motion with realistic physics, including gravity and collisions. This isolates the model's embodied action intelligence from execution-side disturbances such as balance and motor errors. Building on this framework, the authors introduce HumanCLAW-Bench, comprising 1,218 long-horizon, egocentric find-navigate-interact episodes across 41 indoor scenes. Nine state-of-the-art VLMs were evaluated, and none could solve the benchmark; the best model achieved only a 16.8% success rate. Notably, object recognition is not the bottleneck. The key deficiency is embodied self-awareness: current VLMs cannot track their own bodies, determine where they are, whether they have reached a target, or whether they have collided with obstacles. Paper: arXiv 2607.27180.

Paper Overview

Research Area: Computer Vision (CV) Authors: Siyao Li, Jiawei Gu, Shuai Liu, Kairui Hu, Zekun Li, Linjie Li, Chengcheng Tang, Po-Chen Wu, Ivan Shugurov, Lingni Ma, Michael Zollhoefer, Sizhe An, Abhay Mittal, Amy Zhao, Ranjay Krishna, Manling Li, Ziwei Liu, Chuan Guo Published: 2026-07-29 arXiv: 2607.27180

Summary

Evaluating whether a vision-language model (VLM) can act through a physical body is challenging: the outcome of an action couples the VLM's decision with motor control. When a task fails, it is hard to tell whether the VLM made a bad choice or the motor controller simply failed to execute it (e.g., losing balance and falling).

HumanCLAW is an evaluation framework that decouples action decision-making from low-level execution. At every step, a harnessed, off-the-shelf VLM issues an atomic skill command, and the command is translated into a sub-second chunk of continuous full-body motion with real physical consequences, including gravity and collisions. The body can therefore act freely in the physical world, while execution-side disturbances—balance and motor errors—are excluded. What remains measurable is the model's action intelligence: its moment-to-moment choice of what the body should do next.

HumanCLAW-Bench

  • 41 indoor scenes
  • 1,218 long-horizon, egocentric find-navigate-interact episodes
  • Nine state-of-the-art VLMs evaluated
  • Key Findings

  • No tested VLM can solve the benchmark; the best model reaches only a 16.8% success rate.
  • Identifying targets is not the bottleneck.
  • What current VLMs lack is embodied self-awareness: they cannot track their own body, judge where they are, whether they have reached a goal, or whether they have hit an obstacle.

Original Abstract (excerpt)

> Evaluating whether a vision-language model (VLM) can act through a physical body is challenging. The outcome of an action couples the VLM's decision with motor control. When a task fails, it is hard to tell whether the VLM made a bad choice or the motor controller simply failed to execute it, e.g., losing balance and falling. In this work, we introduce HumanCLAW, an evaluation framework that decouples action decision-making from low-level execution...

---

Full paper: arXiv 2607.27180

Tags

#humanclaw#vision-language-models#embodied-ai#benchmark#computer-vision#full-body-control#arxiv

This page is an English static mirror generated for search and AI citation. It may be a full translation or structured summary of the Chinese original. Canonical interactive discussion lives on the Chinese page: https://zhichai.net/topic/178503826