Smarter Models Suffer More From Anonymization: Counterintuitive Findings on LLM Performance

> On the Impact of Anonymization on the Performance of Large Language Models > arXiv: 2609.11335 | Deußer, Hahnbück, Sparrenberg, Uelwer et al. > Five mainstream LLMs × eleven benchmarks × original vs. anonymized input

A Striking Result

Imagine building a medical QA system. For privacy compliance, you replace all person, location, and organization names in the input with placeholders like [PERSON_1], [ORG_2]. Intuitively this shouldn't matter—a GPT-4o-class model should understand that "Patient A is allergic to drug B" and "Zhang San is allergic to penicillin" mean the same thing.

Wrong.

The paper finds: the stronger the model, the larger the performance drop caused by anonymization. Capable models like Qwen2.5-72B and GPT-4o mini suffer the most, while smaller models are less affected.

It's like an experienced doctor who habitually asks about a patient's workplace and family history—strip away all that context, and the veteran is more disoriented than the intern.

Experimental Setup: Five Models × Eleven Benchmarks

The researchers evaluated five mainstream language models:

• Qwen2.5-72B (representative large model)
• GPT-4o mini (commercial closed-source representative)
• Three smaller open-source models

Across eleven benchmarks, each model was run on both original and anonymized inputs. Two anonymization strategies were tested: reversible (replacing names with unique placeholders like [PERSON_1], preserving entity distinctness) and irreversible (deletion or generic [REDACTED] tokens).

Three Counterintuitive Findings

1. Stronger Models Fall Harder

The impact of anonymization is not uniform. Qwen2.5-72B and GPT-4o mini show the largest performance drops. The authors speculate that stronger models learn to exploit entity information more deeply during training—they don't treat "Zhang San" as a mere symbol but link it to a web of contextual knowledge. Once entities are masked, those associations break.

This reveals an overlooked phenomenon: part of model capability comes from implicitly memorized inter-entity relations, not pure abstract reasoning. The stronger the model, the deeper the dependence.

2. TruthfulQA Improves After Anonymization

The most puzzling result. TruthfulQA tests whether models propagate common misconceptions (e.g., "Do vaccines cause autism?"). After anonymization, performance on this benchmark improved.

The explanation: many TruthfulQA traps rely on specific entities (misstatements attributed to particular people) to bait the model. Anonymization removes these entity cues, so the model isn't led astray and falls back on more general factual judgment.

This suggests some of a model's TruthfulQA errors are entity-triggered reflexes—seeing a certain name conditionally evokes related misinformation. Anonymization accidentally severs that reflex.

3. Retrieval Tasks Suffer "Catastrophic Decline"

In stark contrast, retrieval-grounded tasks like RGB—answering questions based on input documents—collapse after anonymization. The reason is direct: these tasks require matching "entity X mentioned in the document" with a question about X. After anonymization, "Zhang San" in both becomes [PERSON_1], but the model can no longer link them because the placeholders have lost semantic entity information.

Two Fixes: One Works, One Doesn't

Reversible vs. irreversible anonymization: Reversible anonymization (preserving entity uniqueness, e.g., [PERSON_1], [PERSON_2]) significantly outperforms irreversible redaction. Models can use the information that "these are two distinct entities" even without knowing who they are.

Explicitly prompting that input is anonymized: Telling the model in the prompt that "the text has been anonymized; [PERSON_1] denotes a specific person's name" produced no discernible benefit. The model knows the input is anonymized but cannot compensate for the loss.

This is important: it shows the problem isn't superficial confusion about placeholders—the model's internal reasoning pathways that depend on entity semantics are severed, and prompt-level hints cannot repair that.

What This Means

The core insight: anonymization is not a one-click switch but an engineering problem that must be co-designed with the model and the task.

For privacy-mandatory domains—healthcare, legal, finance—this means: 1. Never assume large models are "immune" to anonymization—quite the opposite: the stronger the model, the larger the potential performance loss 2. Reversible anonymization is the better choice—preserve entity uniqueness even without preserving identities 3. Be especially careful with retrieval tasks—they may effectively fail under anonymization 4. Fact-checking tasks may unexpectedly benefit—the TruthfulQA improvement hints that some misinformation propagation is entity-triggered

A Deeper Question

The paper leaves an open question worth pondering: if stronger models depend more on entity information, is that dependence "real intelligence" or "advanced pattern matching"?

A model that truly understood "Patient A is allergic to drug B" shouldn't change its judgment based on whether A is "Zhang San" or [PERSON_1]. But the experiments show stronger models depend more on concrete information. Perhaps much of current LLM capability is built on memorized entity relations rather than genuine abstract reasoning.

Anonymization inadvertently became a probe: it measured the ratio of "entity-dependent" to "abstraction-dependent" capability. The result—the entity-dependent share is far larger than we assumed.

---

Paper link: arxiv.org/abs/2609.11335

Code: No public code repository provided by the authors.