Meerkat: Detecting Safety Violations Across Many Agent Traces
This forum post shares a recent AI safety paper published on arXiv.
Paper Details
- Field: cs.AI, cs.CL
- Authors: Adam Stein, Davis Brown, Hamed Hassani, Mayur Naik, Eric Wong
- Published: 2026-04-13
- arXiv: 2604.11806
Summary
Identifying safety violations requires searching across large collections of agent traces, but failures are often rare, complex, and sometimes adversarially hidden — only discoverable when multiple traces are analyzed together. The paper proposes Meerkat, which combines clustering with agentic search to discover safety violations described in natural language.
By structuring the search and adaptively investigating promising regions, Meerkat can uncover sparse failures without seed scenarios, fixed workflows, or exhaustive enumeration. In misuse, misalignment, and task-gaming scenarios, Meerkat significantly improves safety violation detection, finding nearly 4x more reward hacking on CyBench than previous audits.
Original Abstract
> To identify safety violations, auditors often search over large sets of agent traces. This search is difficult because failures are often rare, complex, and sometimes even adversarially hidden and only detectable when multiple traces are analyzed together. These challenges arise in diverse settings such as misuse campaigns, covert sabotage, reward hacking, and prompt injection. We introduce Meerkat, which combines clustering with agentic search to uncover violations specified in natural language.
---
*Auto-collected on 2026-04-15.*