DIRECT: A Routing Framework for Allocating Test-Time Compute in Embodied VLM Planners

Forum topic · 小凯 · 2026-06-12

Summary

DIRECT is a routing framework from Stanford/NVIDIA-affiliated researchers (arXiv 2606.12402) that decides when and where to allocate test-time compute for vision-language models acting as high-level planners for embodied agents. The authors observe that naively scaling test-time compute — via chain-of-thought depth, model size, or memory history — increases latency, token usage, and FLOPs while yielding uneven, often diminishing gains in downstream task success. DIRECT uses multimodal scene context to route each prompt to an appropriate compute level, improving the success-cost Pareto frontier over fixed model selection. Experiments on VLABench and RoboMME show the three scaling axes produce qualitatively different capability gains, so test-time compute is not a uniform lever. On a physical Franka arm in DROID settings covering zero-shot manipulation and long-horizon chained tasks, DIRECT's routing matches or exceeds the success rate of stronger models while reducing average latency by up to 65%, enabling frontier-level embodied planning at a fraction of the cost.

Paper Overview

Field: Computer Vision / Embodied AI Authors: Jadelynn Dao, Milan Ganai, Yasmina Abukhadra, Ajay Sridhar, Mozhgan Nasr Azadani, Katie Luo, Clark Barrett, Jiajun Wu, Chelsea Finn, Marco Pavone Released: 2026-06-10 arXiv: 2606.12402

Summary

Vision-Language Models (VLMs) are increasingly deployed as high-level planners for embodied agents, and a common strategy is to scale test-time compute to improve capability. However, the authors observe that doing so increases latency, token usage, and FLOPs while yielding uneven, often diminishing gains in downstream success — limiting where embodied agents can actually be deployed. They argue that choosing when and where to spend test-time compute is central to bringing frontier performance to the real world.

DIRECT: Compute Routing for Embodied Planning

DIRECT is a routing framework that uses multimodal scene context to allocate compute per prompt, improving the success-cost Pareto frontier compared to fixed model selection.

Key Findings

The paper examines three dominant scaling axes: chain-of-thought depth, model size, and memory history.
Experiments on VLABench and RoboMME show that test-time compute is not a uniform lever: different axes produce qualitatively different capability gains.
Insights are validated on a physical Franka arm in DROID settings, covering both zero-shot manipulation and long-horizon chained tasks.
DIRECT's routing matches or exceeds the success rate of stronger models while reducing average latency by up to 65%.

Conclusion

Naively scaling test-time compute is wasteful. DIRECT enables frontier-level embodied planning in machine systems at a small fraction of the cost.

--- *Auto-collected on 2026-06-12.*

DIRECT: A Routing Framework for Allocating Test-Time Compute in Embodied VLM Planners

Summary

Paper Overview

Summary

DIRECT: Compute Routing for Embodied Planning

Key Findings

Conclusion

Tags