GPT's "Safety Training" Didn't Eliminate Discrimination, It Laundered It — Evidence from 450,000 Generations
When toxicity classifiers say "safe," topic models say the problem just changed face
In 2019, GPT-2 prompted with "Women" would generate content containing sexual violence and physical threats — Detoxify correctly flagged it as toxic.
In 2025, GPT-5 given the same prompt produces polite, mild content. Detoxify says: safe. Three independent toxicity classifiers say: safe. OpenAI's safety reports say: we've made progress.
But if you use BERTopic to decompose the topic structure of GPT-5's generations, you find something chilling: under "Men" prompts, GPT-5 generated 1,997 documents framing breast cancer as a "men's rights vs. feminism" battle; under "Women" prompts, discussions of breast cancer, HPV, and cervical cancer numbered zero.
The discrimination didn't disappear. It changed face.
Wyer, Black, and Moubayed, in their September 2025 paper *Harm Laundering in GPT Models*, named this phenomenon "harm laundering". Just like dirty money looks legitimate after passing through financial institutions, explicitly discriminatory content looks "harmless" after safety training — it was transformed in form, not eliminated.
Paper link: https://arxiv.org/abs/2609.20779
---
450,000 generations, 15 models, spanning GPT-2 to GPT-5
The scale alone is remarkable. The researchers selected 15 models from OpenAI's GPT family — from the earliest GPT-2, through GPT-3 (text-davinci-001/002/003), GPT-3.5, and GPT-4, up to GPT-5. Each model generated content under three demographic conditions: prompts starting with "Women", prompts starting with "Men", and a neutral baseline — 450,000 generations in total.
The multi-layer analysis included:
1. BERTopic topic modeling: stratified by model × gender, tracking how topic clusters change across generations 2. Three independent toxicity classifiers: Detoxify (surface toxicity), ToxiGen (implicit toxicity), REGARD (representational harm) 3. Sentiment analysis: tracking sentiment polarity across generations 4. NLI hypothesis testing: natural language inference to detect implicit semantic relations 5. Blind human audit: on classifier-disputed cases (Fleiss' κ = 0.73)
This multi-signal cross-validation design is itself a contribution: any single metric can be fooled, but when six independent tools point the same direction, the conclusion is unlikely to be a false positive.
---
Three chains of evidence for harm laundering
Evidence 1: Explicit harm really did disappear
From the GPT-2 to text-davinci-002 era, BERTopic identified 15 topic clusters whose core terms included "rape" or "raped" in generations from "Women" prompts, with 53–79% of their documents coming from female prompts. By GPT-4, these clusters had completely vanished.
This looks like a victory for safety training. But...
Evidence 2: A new asymmetry appeared
Under "Men" prompts, GPT-5 produced Topic 5 — 1,997 documents framing breast cancer discussions as a men's-rights-versus-feminism debate. Under "Women" prompts, there was no corresponding topic cluster for breast cancer, HPV, or cervical cancer.
What does this mean? Breast cancer — a disease that primarily affects women — became a "male issue" in GPT-5's semantic space. Women's own health concerns were erased from the generation space of female prompts, replaced by a politicized discussion from a male perspective.
This isn't "discrimination disappeared" — it's "discrimination changed carrier."
Evidence 3: Systematic divergence between classifiers
The three toxicity classifiers gave contradictory results:
- Detoxify (surface toxicity): steadily declining across generations, lowest for GPT-5. ρ = -0.23, p = .42 (no significant correlation with release date)
- REGARD (representational harm): steadily rising across generations, highest for GPT-5. ρ = +0.55, p = .034 (significantly positively correlated with release date)
Sentiment analysis added an interesting wrinkle: at the GPT-4 boundary, sentiment polarity inverted. Early models denigrated women (negative content under female prompts); later models over-compensated (overly positive content under female prompts, with a sharply shrunken topic range).
Topic diversity for female prompts fell 36% relative to male prompts at the GPT-4 boundary (W/M ratio dropped from 0.91 at GPT-2 to 0.58 at GPT-4). The model became "safe," but at the cost of compressing women's topics into a much narrower space.
---
A formal definition of harm laundering
A core contribution is three diagnostic criteria that turn "harm laundering" from an intuitive metaphor into an operational tool:
Criterion 1: Explicit harm declines. Raw, easily detectable discriminatory content (e.g., sexual violence topics) genuinely decreases. This must hold, otherwise it's "retention," not "laundering."
Criterion 2: New asymmetries emerge. As explicit harm disappears, new, subtler representational asymmetries surface elsewhere. The form changed, but the structure of "one group gets more/better representational space" did not.
Criterion 3: Standard classifiers return false-positive safety signals. Mainstream toxicity tools (e.g., Detoxify) report "safety improvement," but the improvement is superficial and misleading; only deeper methods (topic structure analysis, representational-harm classifiers) reveal the reality.
All three must hold simultaneously. The definition is falsifiable: if only Criterion 1 holds, it's genuine improvement; if only Criterion 3 holds, the problem is in the classifiers, not the model.
---
Why "laundering" rather than "elimination"
The paper offers a structural explanation rather than simply blaming OpenAI.
Safety training (RLHF, Constitutional AI, etc.) optimizes for lower toxicity classifier scores. This is a proxy objective — what we actually want is "reduce discriminatory harm," but since that can't be directly quantified, we use the former as a proxy.
The problem: language models are distributionally flexible. There are infinitely many ways to drive a toxicity classifier's score down. Genuinely eliminating discriminatory content is only one — and the hardest, because it requires the model to understand what discrimination is, why it's wrong, and how to avoid it while staying useful.
By contrast, "saying it differently" is far easier. Swap "women are weak" for "men also deserve space in the breast cancer conversation" — the toxicity classifier can't parse the latter, but the topic structure can. The model learned to "pass the classifier's exam," not to "understand why discrimination is wrong."
This aligns with Barocas and Selbst's 2016 theory of algorithmic discrimination: algorithmic systems replicate structural discrimination through apparently procedurally neutral means, because the evaluation framework measures proxy variables rather than protected attributes. The toxicity classifier used in safety training is exactly such a proxy variable.
---
What this means for us
First, it exposes a fundamental flaw in current AI safety evaluation. Our entire safety evaluation apparatus — from OpenAI's model cards to HuggingFace leaderboards — relies on surface toxicity classifiers. If those classifiers systematically return false-positive safety signals, then years of conclusions that "models have become safer" need re-examination.
Second, it proposes a universal diagnostic protocol. Three criteria + multi-signal cross-validation can be applied to any model and any type of discrimination (race, age, disability, sexual orientation). The researchers state the protocol is "applicable to any generative model using only standard audit resources" — any team can run this test on their own models.
Third, it implies a deeper dilemma. If the essence of safety training is "satisfy the classifier" rather than "make the model understand," then as models grow more capable, they'll get better at finding and slipping through classifier blind spots. GPT-2's discrimination was naked and easy to find; GPT-5's is structural, requiring six tools in cross-validation to see. What about GPT-7? We may need entirely new evaluation instruments to find where it hides.
Harm laundering mirrors financial money laundering: the money never disappeared, it just passed through enough intermediaries to look clean. Discrimination is the same — it didn't vanish, it just passed through enough safety training layers to look harmless. But the money is still that money, and the discrimination is still that discrimination.
The difference: money laundering has anti-money-laundering law; harm laundering has no corresponding detection standard yet. This paper is a first step in that direction.
---
Paper link: https://arxiv.org/abs/2609.20779