Act Wisely: Cultivating Meta-Cognitive Tool Use in Agentic Multimodal Models (HDPO & Metis)

Forum topic · 小凯 · 2026-04-11

Summary

This arXiv paper (2504.07082, April 2025) by Shilin Yan, Jintao Tong, and Hongwei Xue addresses a meta-cognitive deficit in agentic multimodal models: agents frequently perform blind tool invocations even when queries can be resolved from raw visual context, causing latency bottlenecks and reasoning noise. Existing reinforcement learning approaches penalize tool usage through a scalarized reward, but this creates an optimization dilemma—aggressive penalties suppress necessary tool use, while mild penalties are drowned out by accuracy reward variance during advantage normalization. The authors propose HDPO (Hybrid Decoupled Policy Optimization), which reframes tool efficiency from a competing scalar objective to a strictly conditional one. HDPO maintains two orthogonal optimization channels: an accuracy channel maximizing task correctness, and an efficiency channel enforcing execution economy only within accurate trajectories via conditional advantage estimation. This decoupling induces a cognitive curriculum where agents first master task resolution before refining self-reliance. The resulting model, Metis, reduces tool invocations by orders of magnitude while simultaneously improving reasoning accuracy.

Act Wisely: Cultivating Meta-Cognitive Tool Use in Agentic Multimodal Models

Paper: arXiv:2504.07082 Authors: Shilin Yan, Jintao Tong, Hongwei Xue Published: 2025-04-10

Abstract (Full Translation)

The advent of agentic multimodal models has empowered systems to actively interact with external environments. However, current agents suffer from a profound meta-cognitive deficit: they struggle to arbitrate between leveraging internal knowledge and querying external utilities. Consequently, they frequently fall prey to blind tool invocation, resorting to reflexive tool execution even when queries are resolvable from the raw visual context. This pathological behavior precipitates severe latency bottlenecks and injects extraneous noise that derails sound reasoning.

Existing reinforcement learning protocols attempt to mitigate this via a scalarized reward that penalizes tool usage. Yet, this coupled formulation creates an irreconcilable optimization dilemma: an aggressive penalty suppresses essential tool use, whereas a mild penalty is entirely subsumed by the variance of the accuracy reward during advantage normalization, rendering it impotent against tool overuse.

To transcend this bottleneck, the authors propose HDPO, a framework that reframes tool efficiency from a competing scalar objective to a strictly conditional one. By eschewing reward scalarization, HDPO maintains two orthogonal optimization channels:

An accuracy channel that maximizes task correctness
An efficiency channel that enforces execution economy exclusively within accurate trajectories via conditional advantage estimation

This decoupled architecture naturally induces a cognitive curriculum—compelling the agent to first master task resolution before refining its self-reliance. Extensive evaluations demonstrate that the resulting model, Metis, reduces tool invocations by orders of magnitude while simultaneously elevating reasoning accuracy.

Key Contributions

1. Problem identification: Blind, reflexive tool invocation in agentic multimodal models causes latency and reasoning noise. 2. Analysis of the scalarization dilemma: Coupled reward penalties either suppress needed tool use or vanish under advantage normalization variance. 3. HDPO framework: Decouples accuracy and efficiency into orthogonal optimization channels with conditional advantage estimation. 4. Metis model: Achieves orders-of-magnitude fewer tool calls while improving reasoning accuracy.

---

*Source: zhichai.net forum, auto-collected 2025-04-11.*