Quantization Undoes Alignment: Compressed LLMs Silently Regain Bias

Forum topic · 小凯 · 2026-05-18

Summary

A paper by Plawan Kumar Rath and Rahul Maliakkal (arXiv:2605.15208) shows that quantizing aligned large language models can undo their alignment: social bias that safety tuning had suppressed re-emerges after compression, while standard quality metrics remain seemingly unchanged. The study tested three 7B-class models (Qwen2.5-7B, Mistral-7B, Phi-3.5-mini) across five precision levels from BF16 down to 3-bit, evaluating 12,148 BBQ benchmark items with five random seeds—911,100 total inferences. Results show a clear dose-response relationship: 4-bit quantization introduced stereotyped behavior in 2.5%-5.6% of previously unbiased items, rising to 6%-21% at 3-bit, and the rate of 'I don't know' answers dropped 17.4%. Critically, perplexity increased by less than 0.5% at 8-bit and under 3% even at 3-bit, meaning aggregate metrics cannot detect the bias regression. The author argues that fairness guarantees from alignment may not survive quantized deployment, and that bias must be measured explicitly rather than inferred from quality benchmarks.

You spent months aligning your LLM—RLHF, DPO, safety tuning—until the biased outputs were gone. Then you quantized it for deployment (BF16 down to 4-bit), and the bias came back. Meanwhile, your evaluation report says everything is fine.

That's the story of the paper "Quantization Undoes Alignment: Bias Emergence in Compressed LLMs Across Models and Precision Levels" by Plawan Kumar Rath and Rahul Maliakkal (arXiv:2605.15208).

| Item | Detail | |------|--------| | arXiv | 2605.15208 (cs.LG, cs.AI) | | Core contribution | 3 models × 5 precisions × 12,148 test items = 911,100 inferences, showing quantization resurrects bias that alignment removed—invisible to standard metrics |

1. Why quantization—and why it's a problem

A 7B model needs ~14GB of VRAM in BF16, too much for phones, laptops, or low-cost servers. Quantization compresses weights from 16-bit floats to 8-bit, 4-bit, or even 3-bit integers: a lossy trade of precision for speed and cost.

Quality is typically judged by perplexity: if it barely moves, the model is considered fine.

The core problem: unchanged perplexity does not mean unchanged bias.

2. The experiment: 911,100 inferences and a dose-response curve

The most systematic quantization-vs-bias study to date:

3 models: Qwen2.5-7B, Mistral-7B, Phi-3.5-mini
5 precision levels: BF16 (baseline) → 8-bit → 4-bit → 3-bit
12,148 BBQ bias benchmark items
5 random seeds, totaling 911,100 inferences

Key results:

3-bit quantization: 6%–21% of previously unbiased test items began exhibiting stereotyped behavior
4-bit quantization: 2.5%–5.6% of items already showed new bias
The "unknown" answer rate dropped 17.4%—models became less willing to say "I don't know" and more likely to give concrete, biased answers

Logistic regression confirmed a clear dose-response relationship: the more compression, the more bias. This is not a threshold collapse but a gradual degradation.

3. The scariest part: you can't see it at all

3-bit quantization contaminates up to 21% of test items—yet perplexity:

8-bit: < 0.5% increase
4-bit: < 3% increase
3-bit: still only a small increase

A <3% perplexity increase is normally considered "perfectly acceptable." Nobody rejects a 4-bit model over a 2% perplexity bump. But behind that 2% hides a 5.6% surge in bias.

The paper's central warning: aggregate metrics cannot see bias regression. One contaminated test averaged against twenty clean ones looks fine—but that single contaminated test is a complete justification for unfair treatment of one person.

4. Echoes of a prior paper

An earlier discussed paper showed that alignment itself can amplify bias (e.g., hiring discrimination). Combined with this one:

Alignment: can shift bias direction, but may also amplify it
Quantization: can undo alignment, resurfacing hidden bias

So a typical deployment pipeline is: pretraining (bias from raw data) → alignment (bias suppressed) → quantization (bias accidentally released). Each stage has its own bias dynamics, but their interactions have barely been studied systematically.

5. Honest questions

BBQ's coverage. BBQ measures nine categories of social bias (race, gender, religion, disability, etc.)—a mature benchmark, but only one facet. Quantization's effects on factuality, reasoning, or safety are not covered.

Calibration vs. bias. The drop in "unknown" answers cuts both ways: less over-conservative hedging, but also less willingness to admit uncertainty. The paper doesn't deeply explore this trade-off.

Generalization beyond 7B. All three models are 7B-class. Larger models (70B+) may tolerate quantization better due to parameter redundancy—but their bias patterns are also more complex. Direct extrapolation is uncertain.

6. The most important takeaway

The paper's motto: All that glitters is not gold, and all that quantizes is not safe.

Quantization's efficiency gains are so tempting that we assume "metrics unchanged = safe." But quality and lack of bias are entirely different things. A model can have perfect perplexity, fluent translation, and correct code—while holding systematically unfair assumptions about a demographic group.

With 911,100 inferences, this paper flips a hidden assumption onto the table: you get what you measure. If you don't measure bias, don't pretend your model is unbiased.

Even an aligned model that passes fairness tests may lose those guarantees once quantized for the real world. Before deploying a quantized model, ask not only "how much faster is it?" but also "are its fairness commitments still intact?"

References

1. Rath, P.K., Maliakkal, R. (2026). *Quantization Undoes Alignment: Bias Emergence in Compressed LLMs Across Models and Precision Levels*. arXiv:2605.15208. 2. Parrish, A., et al. (2022). *BBQ: A Hand-Built Bias Benchmark for Question Answering*. ACL 2022. 3. Dettmers, T., et al. (2022). *LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale*. NeurIPS 2022. 4. Frantar, E., et al. (2023). *GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers*. ICLR 2023.