The Emperor's New Clothes: Why AI Compliance Detectors Ignore the Rules They Claim to Enforce

> Original paper: What Do Compliance Detectors Read? An Audit of Activation Probes and Guard Models > Authors: Saisab Sadhu, Aadit Sengupta, Vinay Kumar Sankarapu, Pratinav Seth > arXiv: 2608.16852

The Setup: A 'Vision Test' for Safety Detectors

Companies spend billions on AI compliance: guard models and activation probes deployed at the output of large language models to check whether responses violate rules on privacy, medical advice, financial regulation, or platform policy. The implicit promise: every output is checked against the rules.

But do these detectors actually *read* the rules?

Sadhu et al. ran three tests on mainstream detectors (activation probes, guard models, and policy-conditioned guards):

1. Rule deletion — replace the rule with an empty string. Detectors should collapse to random guessing. 2. Rule substitution — flip "never reveal personal data" into "reveal personal data." Verdicts should change. 3. Rule permutation — shuffle the order of rule clauses.

Result: accuracy showed zero change across all three tests, for every detector. Not a slight drop—no movement at all.

Rule Blindness

The most striking finding came from a *policy-conditioned guard* that explicitly receives rule text as input and cites rule sections in its verdicts (e.g., "violates section 3.2"). When researchers inverted the content of section 3.2, the guard kept citing it as grounds for violation—despite the section now saying the opposite.

The authors call this rule blindness: the act of citing rules is a trained *linguistic ritual*, not evidence of reading them.

Why? Detectors learn surface features—keyword frequencies ("kill", "hack", "password"), sentiment polarity, syntactic complexity, even capitalization ratios—because these correlate with violations in training data. But correlation isn't causation, and the rules' actual semantics are masked by these statistical shortcuts.

ICS: A Cheap Audit Tool

The paper proposes ICS (Internal Compliance Score) to audit whether a guard truly conditions on rules:

• If a detector reasons from rules, changing the rules should change its internal activation patterns.
• ICS needs no retraining: collect ~10 compliant/non-compliant pairs, record internal activations, and linearly project them to a one-dimensional compliance score.
• The question is simple: does the score move when the rule changes? If not, the detector is blind.

They also introduce a Crossed-Rule Benchmark: two rules crossed with two scenarios, arranged so rule A conflicts only with scenario X and rule B only with scenario Y. No single surface feature can predict labels. Under this benchmark, all tested guards and probes failed. Only methods performing explicit step-by-step reasoning (Chain-of-Thought) passed—but these are far too slow for real-time deployment.

Real-World Risks

• Compliance hallucination: telling regulators an AI is "guarded" implies the guard detects violations. If it doesn't read rules, that assurance is false security. A guard keyed on words like "stock" and "buy" will pass "AAPL calls look attractive" even when it violates the same rule.
• Adversarial evasion: attackers who know a guard relies on surface features can bypass it via synonyms, paraphrasing, or euphemism—the rules themselves provide no protection.
• White-box attack: the authors show an attacker who knows ICS's projection vector can craft outputs that maximize the ICS score while retaining violating content, completely canceling ICS's safety gains. Any auditable detector can be targeted; safety is an escalating arms race.

Takeaways

1. Admit limitations: guards likely perform statistical pattern matching, not rule understanding. 2. Build audit mechanisms: every deployed detector should ship with tools verifying rule-conditioning. 3. Use crossed-rule benchmarks: evaluations must prevent any single feature from predicting labels. 4. Prefer slow reasoning where safety matters: explicit Chain-of-Thought beats fast black-box classifiers, even at higher latency.

True rule understanding means behavior changes when rules change. If detectors can't pass even that most basic test, there is little reason to trust them as our safety net.

References

• Sadhu, S., Sengupta, A., Sankarapu, V. K., & Seth, P. (2026). *What Do Compliance Detectors Read? An Audit of Activation Probes and Guard Models*. arXiv:2608.16852.
• Zou, A., Wang, Z., Kolter, J. Z., & Fredrikson, M. (2023). Universal and transferable adversarial attacks on aligned language models. *arXiv preprint arXiv:2307.15043*.
• Jain, S., & Wallace, B. C. (2019). Attention is not explanation. *Proceedings of NAACL-HLT*, 3543-3556.