Hidden Discrimination in Algorithms: Auditing and Mitigating Social Bias in LLM-Generated Code

Forum topic · 小凯 · 2026-05-06

Summary

A Concordia University study (arXiv:2605.00160) challenges the assumption that code generated by large language models is socially neutral. Using the Solar framework, researchers evaluated 343 programming tasks involving human decisions and introduced a Code Bias Score based on metamorphic testing, which compares outputs when only demographic attributes (e.g., gender) change. Generated code showed strong bias signals, with scores peaking at 60.58% across seven tested dimensions. Counterintuitively, chain-of-thought (CoT) prompting amplified bias: extra reasoning space led the model to rationalize stereotypes found in training data, producing more discriminatory if-else logic. To address this, the paper proposes a Fairness Monitor Agent (FMA) architecture that separates fairness auditing from the main code-generation pipeline, avoiding the diffusion of responsibility seen in multi-agent systems. Experiments showed FMA reduced bias by 65.1% while improving functional correctness from 75.8% to 83.97%. The authors argue code fairness must become a hard quality metric in industrial agent deployments.

随着生成式 AI 深入软件开发的核心链条，代码的“中立性假说”正面临严峻挑战。康考迪亚大学团队在最新论文 arXiv:2605.00160 中揭示了一个核心趋势：LLM 生成的算法往往会继承并固化社会偏见。该研究通过 Solar 框架对 343 个涉及人类决策的编程任务进行了系统评估。

1. Measuring Bias: The Variant Testing Model

The study defines a Code Bias Score, which detects output inconsistency under semantically equivalent inputs that differ only in demographic attributes:

\text{Bias\_Score} = \frac{1}{N} \sum_{i=1}^{N} [f(P_i, A_{base}) \neq f(P_i, A_{variant})]

> What is metamorphic testing? > A software testing technique that verifies correctness by checking whether outputs follow expected relations when inputs are changed in specific ways (variants). Here, it is used to detect whether merely changing a demographic attribute (e.g., gender) causes a change in code logic.

Across seven tested dimensions, model-generated code showed high bias correlation, with scores peaking at 60.58%.

2. Mechanism Warning: Why CoT Backfires

A counterintuitive finding: chain-of-thought (CoT) prompting significantly amplified bias. Technical analysis suggests CoT gives the model more "reasoning space," leading it to seek logically plausible justifications for stereotypes in its training corpus. This "rationalized bias" produces more precise but discriminatory if-else conditions.

3. Governance: FMA Audit Layer Integration

The paper proposes a Fairness Monitor Agent (FMA) architecture that decouples fairness verification from the main production pipeline.

Experiments show FMA not only reduced bias by 65.1%, but by eliminating logical conflicts, raised functional correctness from 75.8% to 83.97%.

> Term: Diffusion of Responsibility > A psychological concept: in multi-agent pipelines, if every agent is generically told to "stay fair," each assumes another stage already handles it, degrading overall compliance. FMA solves this via the single-responsibility principle.

Conclusion

Code bias has become a new dimension of software supply chain security. Future automated software engineering must not pursue generation efficiency alone; it must build multi-agent collaboration environments based on the FMA architecture, treating fairness as a hard metric of code quality for industrial-grade agent deployment.

---

Paper Information

Title: Social Bias in LLM-Generated Code: Benchmark and Mitigation
Authors: Fazle Rabbi, et al.
Institutions: Concordia University, York University
arXiv ID: 2605.00160
Published: 2026-05-01
Categories: cs.SE, cs.AI