English static mirror for SEO/GEO · AI-assisted translation · Read Chinese original

VISTA: Fixing GRPO's Reward Degeneracy in GUI Grounding with Multi-View Self-Verified Training

Forum topic · 小凯 · 2026-06-15

Summary

VISTA (Zhejiang University × Ant Group Venus team, arXiv:2606.14579) identifies a fatal blind spot when applying GRPO to GUI grounding: repeated sampling on a fixed screenshot almost always yields all-correct or all-wrong rollouts, so group variance is zero and gradients vanish—fewer than 5% of training groups are effective. VISTA's solution reframes the problem rather than the algorithm: it generates multiple target-preserving crops of the same GUI instance, so the same target appears under different geometric perspectives with naturally varying difficulty, restoring within-group variance. It adds two safeguards: a self-verified gating mechanism that activates a ground-truth anchor only when the model already produces at least one correct rollout, and model-exclusive group statistics that keep the anchor out of baseline computation to prevent supervision leakage. On Qwen3-VL backbones (4B–30B-A3B), VISTA boosts ScreenSpot-Pro by up to +13.3 points, reaching 72.9 SOTA at the 35B-A3B scale, with consistent gains on ScreenSpot-V2, OSWorld-G, and MMBench, plus improved crop robustness (VCR 90.4). Ablations show view-consistent grouping and gated anchoring are complementary, while ungated oracle normalization actively hurts performance.

> Paper: VISTA: View-Consistent Self-Verified Training for GUI Grounding > arXiv: 2606.14579 | June 2026 > Institutions: Zhejiang University × Ant Group Venus team

TL;DR

VISTA uncovers a fatal blind spot of GRPO on GUI grounding: repeated sampling on the same fixed screenshot produces all-correct or all-wrong rollouts ~99% of the time, so group variance is zero and gradients vanish. The fix is surprisingly simple—don't change the algorithm, change the *view*: generate multiple target-preserving crops of the same GUI instance so the model solves the same task under different geometric perspectives, naturally introducing effective variance.

The Problem: GRPO's "Reward Degeneracy"

GUI grounding: given a screenshot + instruction ("click the export button"), output click coordinates. Reward: +1 if the coordinate lands in the target box, else 0.

With standard GRPO advantages computed as \(A_i = (r_i - \mu_G) / (\sigma_G + \epsilon)\):

  • Hard sample (tiny/ambiguous target): all rollouts wrong → \(r = [0,0,0,0]\), \(\mu=0, \sigma=0\) → all advantages 0 ❌
  • Easy sample (obvious target): all rollouts correct → \(r = [1,1,1,1]\), \(\mu=1, \sigma=0\) → all advantages 0 ❌
  • Result: fewer than 5% of training groups are "effective" (mixed correct/incorrect). This isn't a weak model—GRPO's assumption of sufficient within-group variance breaks on GUI grounding because coordinates are tightly bound to screenshot geometry.

    Core Technique: Three Components

    1. View-Consistent Group Sampling

    Instead of resampling on one screenshot, generate K random crop windows that each fully contain the target box B, remap coordinates to the 0–1000 space, and sample n rollouts per view, forming a group of G = K×n rollouts.

    Why it works: the same target has different difficulty under different views—one crop makes the button clear (2/4 correct), another crops the toolbar (3/4), another partially occludes it (1/4). Variance appears naturally.

    Key details:

  • Default crop size 90% of the original; auto-expanded if smaller than the box
  • 20% pass-through probability keeps full-image samples to avoid over-cropping
  • Exact coordinate remapping preserves label correctness
  • 2. Self-Verified Cross-View Anchoring

    A conditional oracle mechanism: the ground-truth anchor (box center) is activated only if the model already produced at least one fully correct prediction in the group. If not, the oracle stays off—hints don't help a model that hasn't learned yet. The anchor serves only as a stability signal and is excluded from GRPO group statistics. Compared with LUFFY (off-policy teacher trajectories) and BREAD (expert prefix insertion), VISTA needs no stronger teacher and doesn't distort the rollout distribution.

    3. Model-Exclusive Group Statistics

    Baselines μ and σ are computed from model rollouts only. The anchor's advantage is \(A^+ = g \times (R_{max} - \mu_m)/(\sigma_m + \epsilon)\), preventing "ground-truth sequences artificially inflating the baseline" supervision leakage.

    Results: +13.3 on ScreenSpot-Pro

    | Model | SSPro | SSV2 | OSWorld-G | |---|---|---|---| | Qwen3-VL-4B | 55.5 | 88.5 | 58.2 | | VISTA-4B | 63.4 | 94.4 | 63.8 | | Qwen3-VL-8B | 52.7 | 91.7 | 54.8 | | VISTA-8B | 65.8 | 95.5 | 62.4 | | Qwen3-VL-30B-A3B | 53.7 | 94.7 | 66.5 | | VISTA-30A3B | 67.0 | 95.2 | 67.1 |

    ScreenSpot-Pro—known for small targets, high density, high resolution—benefits most. Cross-backbone on Qwen3.5: 35B-A3B reaches 72.9 SOTA (68.6 base → 71.7 GRPO → 72.9 VISTA), confirming the gains aren't initialization-dependent.

    Ablations

  • Crop augmentation alone: +0.6 → it's not the augmentation itself
  • Ungated oracle normalization: 57.8 (worse than GRPO's 63.4) → self-verified gating is critical
  • View grouping (+3.3) and gated anchoring (+2.4) are complementary
  • Crop robustness: VISTA's cropped accuracy (96.25) exceeds its full-image accuracy (95.71), with best view-consistency rate (90.40)
  • Why It Works

    1. Problem reframing, not algorithm innovation. VISTA doesn't modify GRPO's formula—it changes GRPO's input. Like giving a chess player variants of the same puzzle instead of one fixed problem, gradients flow again. 2. Self-verification philosophy generalizes: only give hints after the model proves partial competence—applicable to code generation, math reasoning, dialogue alignment. Hints accelerate convergence; they don't replace thinking. 3. GUI grounding's uniqueness: coordinates are strongly bound to geometry, so fixed-view RL almost inevitably degenerates. View variation here is a necessity, not optional augmentation.

    Limitations

  • Target-preserving crops require knowing target locations; open-ended instructions are unaddressed
  • The 20% pass-through portion can still degenerate on full-image views
  • Multimodal extension is nontrivial—medical imaging, autonomous driving, and robotics have different geometry semantics
  • Per-sample preprocessing cost grows with K crops, complicating the data pipeline
  • References

  • Paper: arXiv:2606.14579
  • Baselines: SeeClick, ShowUI, Aria-UI (SFT); GTA1, UI-TARS, Step-GUI (GRPO-based RL); LUFFY, BREAD (oracle-guided GRPO variants)
  • Benchmarks: ScreenSpot-Pro, ScreenSpot-V2, OSWorld, MMBench
> VISTA's core lesson: when RL fails to converge, sometimes it's the problem definition, not the algorithm. On geometry-bound tasks like GUI grounding, a fixed viewpoint is an information blockade for RL. Changing the view lets the same target present different difficulties, and gradients flow again.

Tags

#vista#gui-grounding#grpo#reinforcement-learning#multimodal#qwen#paper-review#ai-research

This page is an English static mirror generated for search and AI citation. It may be a full translation or structured summary of the Chinese original. Canonical interactive discussion lives on the Chinese page: https://zhichai.net/topic/177981369