Paper Review: Detecting Silent LLM Model Swaps with Behavioral Fingerprints

*(English translation of a zhichai.net forum post)*

Imagine booking a renowned specialist at a hospital, waiting three hours, and then discovering—without being able to tell—that an intern is sitting in the office instead. This is the dilemma AI application developers face today: when you call a ChatGPT or Claude API, how do you know the model behind it hasn't been silently swapped, re-quantized, or moved to a different inference engine? Traditional monitoring (server uptime, latency) can't answer that question.

Models Drift Even Without Swaps

A 2024 Stanford study showed that GPT-4 and GPT-3.5 exhibited significant behavioral drift over months—same questions, different styles and varying success rates. For LLM-dependent applications (legal assistants, educational tools), such silent updates are ticking time bombs that never trigger conventional alerts.

Core Idea: A Behavioral Fingerprint

The paper proposes a behavioral fingerprint—an analogy to human fingerprints: unique per model, but collectable purely from the surface (black-box API access), without knowing weights or architecture.

Step 1: A Fixed Set of "Exam Questions"

A fixed, public, model-agnostic prompt set covering creative writing, logical reasoning, code generation, and common-sense QA.

Step 2: Sample Repeatedly

Each prompt is queried many times (30 samples in the paper), because LLMs are probabilistic—the distribution of answers is the model's "handwriting."

Step 3: Compare Distributions, Not Texts

Answers are converted to embeddings, and the energy distance statistic measures distribution differences. Energy distance is non-parametric, symmetric, and equals zero if and only if two distributions are identical (under mild conditions).

Step 4: Statistical Testing

A permutation test assesses significance: if samples truly come from the same distribution, random regrouping should yield similar distances. The resulting p-value quantifies the evidence against "no change."

Step 5: Sequential Evidence Accumulation

Since single tests can misfire, the system periodically computes new fingerprints, converts p-values into a cumulative evidence score, and raises a change alert once a threshold is crossed—like multiple independent witnesses building a case.

What Can It Detect?

Controlled experiments showed the system reliably detects:

• Model family switches (e.g., Qwen → Llama)
• Version upgrades (GPT-3.5 → GPT-4)
• Inference engine changes (vLLM → Transformers)
• Quantization changes (FP16 → INT8) — subtler, but detectable with enough samples
• Temperature tweaks (even 0.7 → 0.6)

Most strikingly, when monitoring multiple cloud providers serving the "same" model (e.g., Kimi-K2), behavioral differences across providers were sometimes larger than within a single provider over time. Endpoints claiming identical models can behave very differently in practice.

Why It Matters

• Developers can verify the model they pay for hasn't been swapped, without trusting provider claims.
• Regulators gain a black-box audit tool requiring only API access—no weights needed.
• Researchers can study how model behavior evolves across time and providers.

Limitations

• Inference stack details (kernels, caching strategies) affect outputs, and in heterogeneous environments their randomness may be hard to distinguish from true model changes.
• Sampling cost—reliable fingerprints require many API calls, a concern for cost-sensitive applications.

Still, this is a meaningful advance in AI observability: knowing "when it changed and by how much" matters as much as knowing how strong a model is. Just as patients deserve to know who is actually in the examination room, developers now have a way to verify their models.

---

Paper info:

• Title: Behavioral Fingerprints for LLM Endpoint Stability and Identity
• arXiv: 2603.19022
• Authors: Jonah Leshin, Manish Shah, Ian Timmis, Daniel Kang
• Institution: Project VAIL, University of Illinois Urbana-Champaign
• Project site: https://arena.projectvail.com