OS-Themis: A Multi-Agent Critic Framework for Scoring GUI Agents

This post introduces OS-Themis, a scalable critic framework for generalist GUI rewards (arXiv:2603.19191) from researchers at USTC, Shanghai AI Laboratory, and NVIDIA. Named after the Greek goddess of justice, it replaces the unreliable single "judge" with a courtroom-style panel of specialized agents.

Key points

• The problem: GUI agents operating phone/desktop interfaces need accurate reward signals for reinforcement learning, but existing approaches all fail:
• *Rule-based scorers* are rigid, can't handle unseen scenarios, and invite reward hacking.
• *Trained reward models* need expensive annotations and degrade out-of-distribution.
• *LLM-as-a-judge* suffers from context loss, low signal-to-noise ratio, and evidence dilution — dozens of minor successes masking one fatal mid-trajectory failure, which RL then reinforces.
• The solution: four collaborating agents in two stages:

| Agent | Role | |-------|------| | Selector | Extract key milestones from long trajectories | | Verifier | Validate each milestone with before/after screenshots | | Reviewer | Audit the evidence chain; demand refinement if milestones are missing or too lenient | | Judge | Issue the final binary verdict from the whole deliberation process |

• Stage 1 — Milestone Verification Module (MVM): decomposes a 50-step trajectory into fine-grained milestones, each with sub-goal, key step, and importance rationale, then verifies them individually.
• Stage 2 — Verdict Calibration Module (VCM): iteratively audits and refines until the Reviewer is satisfied, yielding interpretable, evidence-backed judgments instead of a black-box score.

Results

• OGRBench (OmniGUIRewardBench): the first cross-platform (mobile/web/desktop) ORM benchmark. OS-Themis beats DigiRL and ZeroGUI with relative gains of +18.8%/+7.7% accuracy, +29.6%/+5.1% precision, +16.9%/+13.0% recall, +26.2%/+13.4% F1.
• Online RL on AndroidWorld: +10.3% agent performance when OS-Themis provides the reward signal.
• Trajectory filtering: +6.9% for Qwen3-VL self-training via data validation.
• Ablations: removing any of the four agents degrades performance — collaboration is essential.

Insights

• Multi-agent specialization outperforms a single all-purpose judge ("four specialized agents beat one generalist").
• Process supervision beats outcome-only supervision for real learning.
• Explainable scoring (which milestones failed, with visual evidence) is invaluable for debugging and high-stakes deployment.

Limitations and future work

• Large-scale online RL is still infrastructure-limited; rewards remain binary with reward shaping at an early stage.
• Ethical risks flagged: semantic reward hacking via VLM reasoning loopholes, bias propagation from pretraining, and privacy concerns around high-fidelity screenshots requiring human-in-the-loop checks.
• Future directions: per-milestone rewards, cross-modal extension (speech, robotics), human-in-the-loop judging, and self-improving critic agents.

References

1. Li, Z., Wu, Z., Zhao, Y., et al. (2026). *OS-Themis: A Scalable Critic Framework for Generalist GUI Rewards*. arXiv:2603.19191. 2. Wang, Z., et al. (2025). *DigiRL: Training In-The-Wild Device-Control Agents with Autonomous Reinforcement Learning*. 3. Zhang, C., et al. (2025). *ZeroGUI: Zero-Shot Graphical User Interface Agents via Visual Imitation*. 4. Rawles, C., et al. (2024). *AndroidWorld: A Dynamic Benchmark for Environment-Centric Android Agents*. 5. Ouyang, L., et al. (2022). *Training language models to follow instructions with human feedback*. NeurIPS 2022.