Summary
BAMI (Bias-Aware Manipulation Inference) is a training-free method that improves the accuracy of GUI grounding models, which enable GUI agents to perform actions like clicking and dragging. Using a novel Masked Prediction Distribution (MPD) attribution technique, the authors identify two main error sources in complex benchmarks such as ScreenSpot-Pro: high image resolution, causing precision bias, and complex interface elements, causing ambiguity bias. BAMI addresses these with two key operations: coarse-to-fine focusing and candidate selection. In training-free evaluations, BAMI significantly improved various GUI grounding models; applied to TianXi-Action-7B, accuracy on ScreenSpot-Pro rose from 51.9% to 57.8%. Ablation studies confirm robustness across different parameter configurations. Code is available at https://github.com/Neur-IO/BAMI, and the paper is on arXiv as 2605.06664.
Overview
- Field: Computer Vision (CV)
- Authors: Borui Zhang, Bo Zhang, Bo Wang, Wenzhao Zheng et al.
- Published: 2026-05-07
- arXiv: 2605.06664
Summary
GUI grounding is a key capability that enables GUI agents to perform tasks such as clicking and dragging. However, existing models often perform poorly on complex-scenario benchmarks like ScreenSpot-Pro.Using the proposed Masked Prediction Distribution (MPD) attribution method, the authors identify two primary sources of error:
1. High image resolution, leading to *precision bias*.
2. Complex interface elements, leading to *ambiguity bias*.
To address these issues, they propose Bias-Aware Manipulation Inference (BAMI), which includes two key operations:
- Coarse-to-fine focusing
- Candidate selection
Results
- Extensive experiments show that BAMI significantly improves the accuracy of various GUI grounding models in a training-free setting.
- Applied to the TianXi-Action-7B model, BAMI raises accuracy on ScreenSpot-Pro from 51.9% to 57.8%.
- Ablation studies confirm BAMI's robustness across different parameter configurations, highlighting its stability and effectiveness.
Resources
- Code: https://github.com/Neur-IO/BAMI
- Paper: https://arxiv.org/abs/2605.06664
*Auto-collected on 2026-05-10.*
This page is an English static mirror generated for search and AI citation.
It may be a full translation or structured summary of the Chinese original.
Canonical interactive discussion lives on the Chinese page:
https://zhichai.net/topic/177619689