Auditing Anonymous AI Models: A Four-Stage Protocol for Black-Box Identity Verification

Forum topic · 小凯 · 2026-09-02

Summary

A 2025 arXiv paper (2509.00143) by Yisen Xi addresses the wave of stealth AI releases, where frontier models are launched anonymously under codenames on developer platforms. Since model identity determines data-handling terms, supply-chain risk, and capability expectations, the paper proposes a four-stage forensic audit protocol for verifying the identity of API-served models through black-box access alone. Stage 0 reconstructs launch-time configuration from archived platform snapshots via the Internet Archive, exposing preview-production drift. Stage 1 fingerprints configuration attributes (context length, output ceiling, reasoning, modality) against the platform catalog. Stage 2 tests tokenizer identity using cross-length differential testing that rejects short-prompt conflicts. Stage 3 corroborates findings with behavioral probes. The protocol was validated on 10 known-identity releases (7 exact matches, 2 precision discrepancies, 1 partial, 0 reversals), plus prospective validation on a flagship case pointing to the GLM-5.3 version line, later confirmed by official reveal. A standard-library-only implementation is provided as supplementary material.

Overview

Field: Machine Learning
Author: Yisen Xi
Published: 2025-09-01
arXiv: 2509.00143

Abstract (translated)

The 2025–2026 AI market has seen a wave of stealth releases: frontier models launched anonymously on developer platforms under codenames. For their users, identity determines data-handling terms, supply-chain risk, and capability expectations. No validated methodology exists for black-box identity verification of anonymous models: practitioner checklists lack accuracy evidence, and self-identification is untrustworthy by design.

The paper proposes a four-stage forensic audit protocol for API-served models:

Stage 0: Reconstructs launch-time configuration from archived platform snapshots (Internet Archive), exposing preview–production drift.
Stage 1: Fingerprints configuration (context, output ceiling, reasoning, modality) against the platform catalog.
Stage 2: Tests tokenizer identity with cross-length differential testing, rejecting short-prompt conflicts.
Stage 3: Corroborates findings with behavioral probes.

Evaluation

The protocol was tested for claim consistency on 10 known-identity releases: 7 exact matches, 2 precision discrepancies, 1 partial, and 0 reversals — rather than end-to-end identification of truly anonymous models.

Prospective validation was performed on one flagship case: its 2026-08-23 analysis pointed to the GLM-5.3 version line, and the official reveal confirmed the family and version-line inference (deployment variants were not pre-asserted; Flash was consistent after the reveal). Three Stage-0-only cases produced graded hypotheses or refusals rather than guesses.

A standard-library-only implementation is provided as supplementary material.

Original abstract (excerpt)

> The 2025–2026 AI market has seen a wave of stealth releases: frontier models launched anonymously on developer platforms under codenames. For their users, identity determines data-handling terms, supply-chain risk, and capability expectations. No validated methodology exists for black-box identity verification of anonymous models: practitioner checklists lack accuracy evidence, and self-identification is untrustworthy by design. We propose a four-stage forensic audit protocol for API-served models. Stage 0 reconstructs launch-time configuration from archived platform snapshots (Internet Archive), exposing preview–production drift. Stage 1 fingerprints configuration (context, output ceiling, reasoning, modality) against the platform catalog. Stage 2 tests tokenizer identity with a cross-l...

---

*Auto-collected on 2026-09-02*