Rule Blindness: Auditing AI Compliance Detectors That Ignore Policy Rules

Forum topic · 小凯 · 2026-08-18

Summary

A Chinese forum post discusses the paper 'What Do Compliance Detectors Read? An Audit of Activation Probes and Guard Models' (arXiv:2608.16852) by Sadhu, Sengupta, Sankarapu, and Seth. The study shows that widely deployed AI compliance detectors—including activation probes, guard models, and policy-conditioned guards—fail basic sensitivity tests: deleting rules, replacing rules with contradictory text, or permuting rule order produces zero change in detection accuracy. The authors call this phenomenon 'rule blindness': detectors rely on surface features (keywords, sentiment, grammar) learned from training data rather than the semantic content of policy rules. Policy-conditioned guards that cite specific rule clauses still cite them even when the clause text has been inverted. The paper proposes the Internal Compliance Score (ICS), a lightweight audit method that checks whether internal activations respond to rule changes, plus a Crossed-Rule Benchmark where no single feature can predict labels; all tested detectors fail it, while only explicit Chain-of-Thought reasoning passes at impractical cost. A white-box attack against ICS fully negates its safety gains. The post warns that guard models may provide false compliance assurances for legal auditing and recommends honest claims, audit mechanisms, crossed-rule testing, and slower reasoning-based detection for critical use cases.

Rule Blindness: Why AI Safety Detectors Ignore the Rules They Claim to Enforce

*This post is a Chinese-language commentary on the paper* What Do Compliance Detectors Read? An Audit of Activation Probes and Guard Models *(Sadhu, Sengupta, Sankarapu & Seth, arXiv:2608.16852).*

Key points

Compliance detectors are widely deployed at LLM output stages to enforce policies on data protection, medical advice, financial regulation, and platform safety. Companies cite them as evidence of legal compliance.
The core finding — "rule blindness": across three sensitivity tests — rule deletion (replacing rules with empty strings), rule substitution (inverting rule meaning), and rule permutation (shuffling clause order) — detection accuracy shows zero change. The detectors are not reading the rules.
Surface-feature reliance: detectors appear to judge based on keyword frequencies (e.g., "kill", "hack", "password"), sentiment polarity, grammatical complexity, and other statistical correlates of violation, rather than rule semantics.
Policy-conditioned guards fail most embarrassingly: they explicitly take rule text as input and cite rule clauses in their verdicts (e.g., "violates clause 3.2"), yet still cite the same clause even after its content has been replaced with the opposite instruction. Citation is a learned linguistic ritual, not comprehension.
Proposed audit tool — ICS (Internal Compliance Score): using ~10 compliance/violation sample pairs and a linear projection of internal activations, ICS tests whether the detector's internal score changes when rules change. No retraining is required.
Crossed-Rule Benchmark: a new evaluation where two rules and two scenarios are designed so that no single surface feature predicts labels. All tested guard models and activation probes fail; only explicit Chain-of-Thought reasoning passes — too slow for production deployment.
White-box attack warning: an attacker who knows ICS's projection vector can craft outputs that maximize the compliance score while retaining violating content, fully negating its safety gains. Any auditable detector can be targeted.

Practical risk illustrated

The post gives a concrete example: a finance AI's guard banning specific stock advice may block "buy Apple stock, it will rise 15%" but pass a paraphrase like "AAPL call options look attractive" — same rule violation, different surface features. Adversaries can exploit synonym substitution and semantic rewriting because the guard never consults the rule text itself.

Authors' recommendations

1. Do not claim guards "understand" rules — they may be doing statistical pattern matching. 2. Pair every deployed detector with audit mechanisms (e.g., ICS) verifying rule sensitivity. 3. Evaluate detectors with crossed-rule benchmarks where single features cannot predict labels. 4. For safety-critical use, prefer explicit reasoning over fast black-box classifiers, even at higher latency.

References

Sadhu, S., Sengupta, A., Sankarapu, V. K., & Seth, P. (2026). *What Do Compliance Detectors Read? An Audit of Activation Probes and Guard Models*. arXiv:2608.16852.
Zou, A., et al. (2023). Universal and transferable adversarial attacks on aligned language models. arXiv:2307.15043.
Jain, S., & Wallace, B. C. (2019). Attention is not explanation. *Proceedings of NAACL-HLT*, 3543–3556.

> If a detector's behavior does not change when the rules change, on what basis should we trust it to protect us?