BAMI: Training-Free Bias Mitigation for GUI Grounding Models

Forum topic · 小凯 · 2026-05-10

Summary

BAMI (Bias-Aware Manipulation Inference) is a training-free method that improves the accuracy of GUI grounding models, which are essential for enabling GUI agents to perform actions like clicking and dragging. Using a novel Masked Prediction Distribution (MPD) attribution technique, the authors identify two main sources of grounding errors on complex benchmarks such as ScreenSpot-Pro: high image resolution (causing precision bias) and complex UI elements (causing ambiguity bias). BAMI addresses these with two key operations: coarse-to-fine focusing and candidate selection. Without any training, the method significantly boosts accuracy across various GUI grounding models—for example, raising TianXi-Action-7B's ScreenSpot-Pro accuracy from 51.9% to 57.8%. Ablation studies confirm robustness across different parameter configurations. Code is available at https://github.com/Neur-IO/BAMI, and the paper is on arXiv at https://arxiv.org/abs/2605.06664.

Paper Overview

Field: Computer Vision
Authors: Borui Zhang, Bo Zhang, Bo Wang, Wenzhao Zheng et al.
arXiv: 2605.06664
Code: https://github.com/Neur-IO/BAMI

Abstract (Translation)

GUI grounding is a key capability that enables GUI agents to perform tasks such as clicking and dragging. However, existing models often underperform on complex benchmarks like ScreenSpot-Pro.

Using the proposed Masked Prediction Distribution (MPD) attribution method, the authors identify two main sources of errors:

1. High image resolution — leading to *precision bias* 2. Complex interface elements — leading to *ambiguity bias*

To address these issues, the paper introduces Bias-Aware Manipulation Inference (BAMI), which consists of two key operations:

Coarse-to-fine focusing
Candidate selection

Extensive experiments show that BAMI significantly improves the accuracy of various GUI grounding models in a training-free setting. For example, applying BAMI to the TianXi-Action-7B model raises its ScreenSpot-Pro accuracy from 51.9% to 57.8%. Ablation studies further confirm the method's robustness across different parameter configurations, highlighting its stability and effectiveness.

Key Takeaways

Training-free: no fine-tuning required to gain accuracy gains
Diagnoses bias sources via MPD attribution
Consistent improvements across multiple grounding models

---

*Auto-collected on 2026-05-10. Original abstract available on arXiv.*