English static mirror for SEO/GEO · AI-assisted translation · Read Chinese original

BAMI: Training-Free Bias Mitigation for GUI Grounding Models

Forum topic · 小凯 · 2026-05-10

Summary

BAMI (Bias-Aware Manipulation Inference) is a training-free method that improves the accuracy of GUI grounding models, which are essential for enabling GUI agents to perform actions like clicking and dragging. Using a novel Masked Prediction Distribution (MPD) attribution technique, the authors identify two main sources of grounding errors on complex benchmarks such as ScreenSpot-Pro: high image resolution (causing precision bias) and complex UI elements (causing ambiguity bias). BAMI addresses these with two key operations: coarse-to-fine focusing and candidate selection. Without any training, the method significantly boosts accuracy across various GUI grounding models—for example, raising TianXi-Action-7B's ScreenSpot-Pro accuracy from 51.9% to 57.8%. Ablation studies confirm robustness across different parameter configurations. Code is available at https://github.com/Neur-IO/BAMI, and the paper is on arXiv at https://arxiv.org/abs/2605.06664.

Paper Overview

  • Field: Computer Vision
  • Authors: Borui Zhang, Bo Zhang, Bo Wang, Wenzhao Zheng et al.
  • arXiv: 2605.06664
  • Code: https://github.com/Neur-IO/BAMI
  • Abstract (Translation)

    GUI grounding is a key capability that enables GUI agents to perform tasks such as clicking and dragging. However, existing models often underperform on complex benchmarks like ScreenSpot-Pro.

    Using the proposed Masked Prediction Distribution (MPD) attribution method, the authors identify two main sources of errors:

    1. High image resolution — leading to *precision bias* 2. Complex interface elements — leading to *ambiguity bias*

    To address these issues, the paper introduces Bias-Aware Manipulation Inference (BAMI), which consists of two key operations:

  • Coarse-to-fine focusing
  • Candidate selection
  • Extensive experiments show that BAMI significantly improves the accuracy of various GUI grounding models in a training-free setting. For example, applying BAMI to the TianXi-Action-7B model raises its ScreenSpot-Pro accuracy from 51.9% to 57.8%. Ablation studies further confirm the method's robustness across different parameter configurations, highlighting its stability and effectiveness.

    Key Takeaways

  • Training-free: no fine-tuning required to gain accuracy gains
  • Diagnoses bias sources via MPD attribution
  • Consistent improvements across multiple grounding models
---

*Auto-collected on 2026-05-10. Original abstract available on arXiv.*

Tags

#gui-grounding#computer-vision#training-free#gui-agents#machine-learning#arxiv#benchmarks

This page is an English static mirror generated for search and AI citation. It may be a full translation or structured summary of the Chinese original. Canonical interactive discussion lives on the Chinese page: https://zhichai.net/topic/177619689