Paper Overview
Research Area: Computer Vision (CV) Authors: Shilin Yan, Jintao Tong, Hongwei Xue Published: 2025-04-10 arXiv: 2504.07927
Summary
The advent of agentic multimodal models has empowered systems to actively interact with external environments. However, current agents suffer from a profound meta-cognitive deficit: they struggle to arbitrate between leveraging internal knowledge and querying external utilities. Consequently, they frequently fall prey to blind tool invocation, resorting to reflexive tool execution even when queries are resolvable from the raw visual context. This pathological behavior precipitates severe latency bottlenecks and injects extraneous noise that derails sound reasoning.
Existing reinforcement learning protocols attempt to mitigate this via a scalarized reward that penalizes tool usage. Yet, this coupled formulation creates an irreconcilable optimization dilemma: an aggressive penalty suppresses necessary tool use, while a mild penalty is entirely drowned out by the variance of accuracy rewards during advantage normalization, leaving it powerless against tool overuse.
The HDPO Framework
To move beyond this bottleneck, the authors propose HDPO, which reframes tool efficiency from a competing scalar objective into a strict conditional objective:
- No reward scalarization: HDPO maintains two orthogonal optimization channels.
- Accuracy channel: maximizes task correctness.
- Efficiency channel: enforces execution economy only within accurate trajectories, via conditional advantage estimation.
Results
Extensive evaluations show that the final model, Metis, reduces tool calls by orders of magnitude while improving reasoning accuracy.
--- *Auto-collected on 2026-04-12*