Auditing Anonymous AI Models: A Four-Stage Protocol for Black-Box Identity Verification

This post summarizes a machine learning paper posted on arXiv on 2025-09-01:

• Paper: Auditing Anonymous AI Models (arXiv:2509.00143)
• Author: Yisen Xi

Key Points

• The 2025–2026 AI market has seen a wave of stealth releases: frontier models launched anonymously on developer platforms under codenames.
• For users, identity determines data-handling terms, supply-chain risk, and capability expectations. No validated methodology exists for black-box identity verification of anonymous models: practitioner checklists lack accuracy evidence, and self-identification is untrustworthy by design.
• The paper proposes a four-stage forensic audit protocol for API-served models:
• Stage 0: Reconstructs launch-time configuration from archived platform snapshots (Internet Archive), exposing preview-production drift.
• Stage 1: Fingerprints configuration (context, output ceiling, reasoning, modality) against the platform catalog.
• Stage 2: Tests tokenizer identity with a cross-length differential test, rejecting short-prompt conflicts.
• Stage 3: Corroborates with behavioral probes.

Evaluation

• Tested claim consistency on 10 known-identity releases: 7 exact, 2 precision differences, 1 partial, 0 reversed — rather than end-to-end identification of anonymous models.
• Prospective validation on one flagship case: its 2026-08-23 analysis pointed to the GLM-5.3 version line, and the official reveal confirmed the family and version-line inferences (deployment variants were not pre-asserted; Flash was consistent after the reveal).
• Three Stage-0-only cases produced graded hypotheses or refusals rather than guesses.

A stdlib-only implementation is provided as supplementary material.

--- *Auto-collected on 2026-09-02.*