CAVEAT: A Benchmark for Robust Computer-Use Agents in Incentive-Misaligned Environments

Forum topic · 小凯 · 2026-09-25

Summary

CAVEAT (arXiv:2609.27273) is a controlled benchmark testing whether computer-use agents (CUAs) preserve user objectives when online environments have misaligned incentives. It spans nine marketplace environments and a taxonomy of eight common steering mechanisms, such as platform product favoritism. Across five model families, agents purchased the user-optimal product in 78.6% of matched-control episodes but only 17.3% when steering mechanisms were enabled. Larger models and more reasoning improved robustness, yet failures persisted. Trajectory analysis and ablations identified three failure stages: agents distort user priorities, prematurely narrow the consideration set, and commit before resolving decision-relevant evidence. Guided by this diagnosis, the authors built CAVEAT-Harness targeting these failure modes, raising the user-optimal purchase rate by 55.0%. Targeted post-training further improved smaller open-source models. The work establishes incentive robustness as a distinct challenge for agentic AI, offering a diagnostic framework and showing that targeted interventions substantially improve fidelity to user goals in commercial online environments.

Paper Overview

Field: NLP Authors: Yuxuan Li, Will Epperson, Wesley Deng, Zezhou Huang Published: 2026-09-25 arXiv: 2609.27273

Introduction

Computer-use agents (CUAs) increasingly act on behalf of users online. What happens when the environments they operate in have incentives that do not align with the user's? In online marketplaces, for example, platforms may favor some products over others, potentially steering agents away from the user's objective. Existing CUA benchmarks cover cooperative settings or explicit attacks, but do not test whether agents preserve user objectives when the environment itself has a stake in the outcome.

Key Contributions

CAVEAT benchmark: a controlled benchmark spanning nine marketplace environments and a taxonomy of eight common steering mechanisms.
Key finding: across five model families, agents purchase the user-optimal product in 78.6% of matched-control episodes, but only 17.3% when steering mechanisms are enabled.
Scaling helps, but not enough: larger models and more reasoning improve robustness, yet substantial failures persist.

Three Failure Modes

Trajectory analysis and targeted ablations identify three stages through which steering enters agent decisions:

1. Agents distort user priorities. 2. Agents prematurely narrow the considered set of alternatives. 3. Agents commit before resolving decision-relevant evidence.

CAVEAT-Harness and Mitigations

Guided by this diagnosis, the authors develop CAVEAT-Harness, which directly targets these failure modes and improves the user-optimal purchase rate by 55.0%. Targeted post-training further improves smaller open-source models.

Conclusion

These results establish "incentive robustness" as a distinct challenge for delegated agents, diagnose how steering failures occur, and show that targeted interventions can substantially improve agent fidelity to user goals.

---

*Auto-collected on 2026-09-25*