Overview
Research area: Computer Vision (CV) Authors: Borui Zhang, Bo Zhang, Bo Wang Published: 2026-05-06 arXiv: 2505.03484
Summary
GUI grounding is a critical capability for enabling GUI agents to execute tasks such as clicking and dragging. However, in complex scenarios like the ScreenSpot-Pro benchmark, existing models often suffer from suboptimal performance.
Using the proposed Masked Prediction Distribution (MPD) attribution method, the authors identify two primary sources of errors:
- Precision bias — caused by high image resolution
- Ambiguity bias — caused by intricate interface elements
- Extensive experiments demonstrate that BAMI significantly enhances the accuracy of various GUI grounding models without any retraining.
- Applied to the TianXi-Action-7B model, BAMI improves accuracy on the ScreenSpot-Pro benchmark from 51.9% to 57.8%.
- Ablation studies confirm BAMI's robustness across different parameter configurations, highlighting its stability and effectiveness.
To address these challenges, the paper introduces Bias-Aware Manipulation Inference (BAMI), which incorporates two key manipulations:
1. Coarse-to-fine focus 2. Candidate selection
These effectively mitigate the identified biases in a fully training-free setting.
Results
*Auto-collected on 2026-05-09.*