Paper Overview
Field: AI/ML Authors: Zhaoyuan Huang, Tianjie Ju, Pengzhou Cheng Published: 2026-09-06 arXiv: 2509.00009
Summary
Graphical user interface (GUI) agents are increasingly used to execute natural-language instructions on user interfaces, yet real users may issue infeasible instructions due to benign mistakes. A reliable agent should not only know how to act, but also when not to act. This work introduces CONFLICTGUI, a benchmark covering instruction-internal conflicts and instruction-GUI context conflicts to study conflict-aware termination.
The evaluation reveals severe execution-biased overcompliance: agents that perform well on feasible tasks often continue to execute blindly under conflicting instructions.
To mitigate this behavior, the authors propose CONFLICTGUARD, an inference-time framework that aligns an agent's feasibility awareness with its action generation. CONFLICTGUARD contains two coupled components:
- Feasibility verification protocol: guides the agent to assess instruction logic and GUI-side evidence before acting.
- Conditional action conditioning mechanism: shifts the agent from overcompliant execution toward termination-oriented behavior.
- New benchmark CONFLICTGUI targets two conflict types: instruction-internal conflicts and instruction-GUI context conflicts.
- Strong agents still overcomply — they blindly execute infeasible/conflicting instructions.
- CONFLICTGUARD is inference-time only, requiring no retraining.
- Improves conflict-task success rates across five agents without degrading standard task performance.
Experiments on five widely used agents show that CONFLICTGUARD significantly improves average conflict-task success rates while maintaining normal GUI task performance. These results validate that lightweight inference-time intervention can substantially improve GUI agents' ability to recognize inappropriate execution scenarios and avoid unnecessary actions.