Quantization Undoes Alignment: How Compressed LLMs Quietly Regain Bias

Forum topic · 小凯 · 2026-05-18

Summary

A detailed Chinese-language forum review discusses the arXiv paper "Quantization Undoes Alignment: Bias Emergence in Compressed LLMs Across Models and Precision Levels" (arXiv:2605.15208) by Rath and Maliakkal. The study runs 911,100 inferences across three models (Qwen2.5-7B, Mistral-7B, Phi-3.5-mini) at five precision levels (BF16 to 3-bit) on 12,148 BBQ bias benchmark items with five random seeds. Findings show quantization reintroduces social bias in aligned models: 3-bit quantization causes 6%-21% of previously unbiased items to exhibit stereotyped behavior, 4-bit causes 2.5%-5.6%, and "unknown" response rates drop 17.4%. A logistic regression confirms a dose-response relationship: more compression yields more bias. Critically, perplexity barely changes (<0.5% at 8-bit, <3% at 4-bit), meaning standard aggregate quality metrics cannot detect the regression. The author notes limitations (BBQ's scope, calibration trade-offs, generalization beyond 7B models) and warns that model quality and fairness must be measured separately before deploying quantized models.

Overview

This post reviews the arXiv paper "Quantization Undoes Alignment: Bias Emergence in Compressed LLMs Across Models and Precision Levels" by Plawan Kumar Rath and Rahul Maliakkal (arXiv:2605.15208).

The premise: you align a large model with RLHF, DPO, and safety tuning, it stops producing biased outputs — then you quantize it for deployment (BF16 down to 4-bit), and the bias returns. Meanwhile, your standard evaluation report shows everything is fine.

Why Quantization Is a Problem

A 7B-parameter model at BF16 needs roughly 14GB of memory — too large for phones, laptops, or low-cost servers. Quantization compresses weights to 8-bit, 4-bit, or even 3-bit integers. It is lossy compression: you trade precision for speed and cost. Quality is usually measured by perplexity — but unchanged perplexity does not mean the bias hasn't returned.

The Experiment: 911,100 Inferences

The most systematic quantization-bias study to date:

3 models: Qwen2.5-7B, Mistral-7B, Phi-3.5-mini
5 precision levels: BF16 (baseline) → 8-bit → 4-bit → 3-bit
12,148 BBQ bias benchmark items
5 random seeds, totaling 911,100 inferences

Key results:

3-bit quantization: 6%–21% of previously unbiased test items begin exhibiting stereotyped behavior
4-bit quantization: 2.5%–5.6% of items show new bias
"Unknown" response rate drops 17.4% — the model is less willing to say "I don't know" and more likely to give specific, biased answers

Logistic regression confirms a clear dose-response relationship: the more compression, the more bias. It's a gradual degradation, not a threshold collapse.

The Scariest Part: You Can't See It

Despite up to 21% of test items being contaminated at 3-bit:

8-bit: perplexity increase < 0.5%
4-bit: perplexity increase < 3%
3-bit: only a small increase

A < 3% perplexity increase is usually considered perfectly acceptable. Nobody rejects a 4-bit quantized model over a 2% perplexity bump — yet behind that 2% hides a 5.6% bias outbreak.

The paper's core warning: you evaluate model quality with aggregate metrics, but bias changes are invisible at the aggregate level. One contaminated test averaged over twenty unaffected ones looks fine — but that one contaminated test is the full justification for one person being treated unfairly.

Relation to Prior Work

Connecting to a previous round's paper on "AI alignment amplifying hiring discrimination":

Alignment can change the direction of bias, and may also amplify it
Quantization can undo alignment's effects, letting hidden bias resurface

A typical deployment pipeline is: pretraining (bias from raw data) → alignment (suppresses bias) → quantization (accidentally liberates bias). Each step has its own bias interventions, but their interactions have barely been studied.

Honest Questions

1. BBQ's coverage: BBQ measures nine categories of social bias (race, gender, religion, disability, etc.) — a mature but partial view. Effects on other model deficits (factual accuracy, reasoning, safety) aren't covered. 2. Calibration vs. bias: The drop in "unknown" responses cuts both ways — less over-conservatism, but also less willingness to admit uncertainty. The balance isn't deeply discussed. 3. Generalization from 7B: All three models are 7B-scale. Larger models (70B+) may tolerate quantization better due to parameter redundancy, but their bias patterns are more complex.

Key Takeaway

The paper's motto: all that glitters is not gold, and all that quantizes is not safe.

"Quality" and "absence of bias" are entirely different concepts. A model can have perfect perplexity, fluent translation, correct code — and still hold systematically unfair assumptions about a demographic group. If you don't measure bias, don't pretend your model doesn't have it. Before deploying a quantized model, ask not just "how much faster is it" but "do its fairness guarantees still hold."

References

1. Rath, P.K., Maliakkal, R. (2026). Quantization Undoes Alignment: Bias Emergence in Compressed LLMs Across Models and Precision Levels. arXiv:2605.15208. 2. Parrish, A., et al. (2022). BBQ: A Hand-Built Bias Benchmark for Question Answering. ACL 2022. 3. Dettmers, T., et al. (2022). LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale. NeurIPS 2022. 4. Frantar, E., et al. (2023). GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers. ICLR 2023.