Auditing Anonymous AI Models: A Four-Stage Protocol for Black-Box Identity Verification

Forum topic · 小凯 · 2026-09-02

Summary

A 2025 arXiv paper (2509.00143) by Yisen Xi proposes a four-stage forensic audit protocol for verifying the identity of anonymously launched AI models served via APIs. Amid a wave of stealth releases where frontier models appear under codenames, identity determines data-handling terms, supply-chain risk, and capability expectations, yet no validated black-box verification method previously existed. Stage 0 reconstructs launch-time configuration from Internet Archive platform snapshots to expose preview-production drift; Stage 1 fingerprints configuration (context length, output ceiling, reasoning, modality) against the platform catalog; Stage 2 tests tokenizer identity using cross-length differential tests; Stage 3 corroborates with behavioral probes. Evaluated on 10 known-identity releases, the protocol achieved 7 exact matches, 2 precision discrepancies, 1 partial, and 0 contradictions. Prospective validation included a flagship case pointing to the GLM-5.3 lineage, later confirmed by official disclosure. A standard-library-only implementation is provided.

Overview

Field: Machine Learning Author: Yisen Xi Published: 2025-09-01 arXiv: 2509.00143

Motivation

The 2025–2026 AI market has seen a wave of stealth releases: frontier models launched anonymously on developer platforms under codenames. For users, identity determines data-handling terms, supply-chain risk, and capability expectations. However, no validated methodology exists for black-box identity verification of anonymous models: practitioner checklists lack accuracy evidence, and self-identification is untrustworthy by design.

The Four-Stage Forensic Audit Protocol

Stage 0 — Archival reconstruction: Reconstructs launch-time configuration from archived platform snapshots (Internet Archive), exposing preview–production drift.
Stage 1 — Configuration fingerprinting: Fingerprints configuration (context length, output ceiling, reasoning, modality) against the platform catalog.
Stage 2 — Tokenizer identification: Tests tokenizer identity with cross-length differential tests, rejecting short-prompt conflicts.
Stage 3 — Behavioral corroboration: Uses behavioral probes to corroborate findings.

Evaluation

The protocol was tested for claim consistency on 10 known-identity releases, yielding:

7 exact matches
2 precision discrepancies
1 partial match
0 contradictions

Rather than claiming end-to-end identification of anonymous models, the evaluation focuses on consistency of claims where ground truth is known.

Prospective Validation

Flagship case: An analysis on 2026-08-23 pointed to the GLM-5.3 version line; official disclosure confirmed the family and version-line inference (deployment variants were not pre-asserted; a "Flash" designation was consistent post-revelation).
Three Stage-0-only cases: The protocol produced graded hypotheses or refusals rather than guesses when evidence was limited.

Availability

A standard-library-only implementation is provided as supplementary material.

---

*Auto-collected on 2026-09-02*