SpatialClaw: Rethinking the Action Interface for Agentic Spatial Reasoning

Forum topic · 小凯 · 2026-06-13

Summary

SpatialClaw (arXiv 2506.10665) is a training-free framework for agentic 3D/4D spatial reasoning built on vision-language models. The authors, Seokju Cho, Ryo Hachiuma, and Abhishek Badki, observe that existing spatial agents are limited by their action interfaces: single-pass code execution commits to a full analysis strategy before seeing any intermediate results, while structured tool-call interfaces offer little flexibility for composing operations. SpatialClaw instead uses code as the action interface, maintaining a stateful Python kernel preloaded with input frames plus perception and geometry primitives. The VLM-based agent writes one executable code unit per step, conditioning each step on all previous textual and visual outputs, enabling flexible composition of perception results and task-adaptive analysis. Evaluated on 20 spatial reasoning benchmarks spanning static and dynamic 3D/4D tasks, SpatialClaw achieves 59.9% average accuracy, outperforming the nearest spatial agent by +11.2 points, with consistent gains across six VLM backbones from two model families and no benchmark- or model-specific adaptation.

Paper Overview

Field: Computer Vision
Authors: Seokju Cho, Ryo Hachiuma, Abhishek Badki
Released: 2025-06-13
arXiv: 2506.10665

Abstract

Spatial reasoning — the ability to determine where objects are, how they relate, and how they move in 3D — remains a fundamental challenge for vision-language models (VLMs). Tool-augmented agents attempt to address this by augmenting VLMs with specialist perception modules, yet their effectiveness is bounded by the action interface through which those tools are invoked. In this work, the authors study how the design of this interface shapes the agent's capacity for open-ended spatial reasoning.

Existing spatial agents either employ single-pass code execution, which commits to a full analysis strategy before any intermediate result is observed, or rely on a structured tool-call interface that often offers less flexibility for freely composing operations or tailoring the analysis to each task. Both designs provide limited flexibility for open-ended, complex 3D/4D spatial reasoning.

SpatialClaw

The authors propose SpatialClaw, a training-free spatial reasoning framework that adopts code as the action interface. SpatialClaw maintains a stateful Python kernel preloaded with the input frames and a suite of perception and geometry primitives. The VLM-powered agent writes one executable unit per step, conditioned on all previous outputs, allowing it to flexibly compose and manipulate perception results and adapt its analysis to intermediate textual and visual observations as well as the needs of each question.

Results

Evaluated on 20 spatial reasoning benchmarks spanning a broad range of static and dynamic 3D/4D spatial reasoning tasks.
Achieves 59.9% average accuracy, exceeding the most recent spatial agent by +11.2 percentage points.
Delivers consistent gains across six VLM backbones from two model families, without any benchmark- or model-specific adaptation.

---

*Auto-collected on 2026-06-13.*