Paper Overview
Research Area: Computer Vision Authors: Borui Zhang, Bo Zhang, Bo Wang Published: 2026-05-06 arXiv: 2505.03484
Summary
GUI grounding is a critical capability for enabling GUI agents to execute tasks such as clicking and dragging. However, in complex scenarios like the ScreenSpot-Pro benchmark, existing models often suffer from suboptimal performance.
Using the proposed Masked Prediction Distribution (MPD) attribution method, the authors identify two primary sources of errors:
- High image resolution — leading to *precision bias*
- Intricate interface elements — resulting in *ambiguity bias*
- BAMI significantly enhances the accuracy of various GUI grounding models in a training-free setting.
- Applied to the TianXi-Action-7B model, BAMI raises accuracy on ScreenSpot-Pro from 51.9% to 57.8%.
- Ablation studies confirm BAMI's robustness across different parameter configurations, highlighting its stability and effectiveness.
Method
To address these challenges, the paper introduces Bias-Aware Manipulation Inference (BAMI), which incorporates two key inference-time manipulations:
1. Coarse-to-fine focus 2. Candidate selection
These operations effectively mitigate the identified biases without requiring any model retraining.
Results
*Auto-collected on 2026-05-09*