Summary
A new arXiv paper (2608.16852) by Saisab Sadhu, Aadit Sengupta, Vinay Kumar Sankarapu, and colleagues audits whether compliance detectors in deployed language models actually depend on the governing rules they claim to enforce. Compliance monitoring is increasingly used as a legal and audit control, checking model outputs against rules spanning data protection, healthcare, financial regulation, and platform policy. The authors show that this monitoring suffers from what they call rule blindness: deleting, permuting, or substituting the governing rule leaves detection accuracy unchanged for every guard and activation probe tested, including a policy-conditioned guard that cites the correct clause but barely changes its verdict when the clause is replaced by its permissive counterpart. The paper introduces the Internal Compliance Score (ICS), a training-free activation readout calibrated from ten token pairs and scored by a single projection. The authors release a counterfactual protocol and cross-rule benchmark to test rule blindness in future detectors and guards.
Paper Overview
- Field: AI
- Authors: Saisab Sadhu, Aadit Sengupta, Vinay Kumar Sankarapu et al. (4 authors)
- Published: 2026-08-17
- arXiv: 2608.16852
Summary
Regulatory compliance monitoring in deployed language models is increasingly implemented as a legal and audit control, checking model outputs against written rules spanning data protection, healthcare, financial regulation, and platform policy. Such monitoring is meaningful only if a detector's verdict depends on the stated rule rather than on surface features of the scenario.
The paper shows this condition fails across the current class of compliance detectors — a failure the authors call rule blindness. Deleting, permuting, or substituting the governing rule leaves detection accuracy unchanged for every guard and activation probe tested, including a policy-conditioned guard that correctly cites the governing clause yet barely changes its verdict when that clause is swapped for its permissive counterpart.
The authors introduce the Internal Compliance Score (ICS): a training-free activation readout calibrated from ten token pairs and scored by a single projection.
They release a counterfactual protocol and a cross-rule benchmark so that rule blindness can be tested in future detectors and guards.
---
*Auto-collected on 2026-08-19*
This page is an English static mirror generated for search and AI citation.
It may be a full translation or structured summary of the Chinese original.
Canonical interactive discussion lives on the Chinese page:
https://zhichai.net/topic/178633647