This forum post reviews Moral Entropy: Auditing Bias and Uncertainty in Moral Judgment (arXiv:2609.21992) by Maciej Skorski (University of Luxembourg). Code: https://github.com/maciejskorski/moral-entropy
Key points
- Motivation via a parable: three judges score a dive; one always scores high. The any-annotator rule (logical OR) amplifies that bias, while majority voting filters it out but also discards legitimate positive scores from the other judges. The paper quantifies "judge bias" in a Bayesian framework and shows that all mainstream moral-label aggregation rules are systematically biased — in different directions.
- Background: Moral Foundations Theory divides moral judgment into Care, Fairness, Loyalty, Authority, and Sanctity. Corpora are labeled by multiple annotators, then aggregated — commonly via any-annotator (any annotator marks a dimension → label = True) or majority voting. Disagreement can reflect genuine moral divergence or annotator bias, and current pipelines collapse both into one vote.
- The framework: Moral Entropy keeps the full posterior P(label | annotations) instead of a hard 0/1 label. It decomposes posterior entropy into:
- Aleatoric uncertainty — irreducible disagreement inherent to ambiguous texts;
- Epistemic uncertainty — reducible disagreement from annotator bias, sparse annotation, or noise. It also learns a per-annotator confusion matrix θ_j, discounting labels from annotators with poor reliability on a given dimension.
- Audit results (106,627 texts, 250,000+ annotations across MFTC, MFRC, and an extended corpus, 15 discourse domains):
- The any-annotator rule disagrees with the Bayesian posterior on ~30% of items — nearly all false positives at dataset level, but at foundation level errors flip (MFTC: 19.9% FPR vs 38.9% FNR).
- Majority/two-vote rules nearly eliminate false positives (MFTC: 3.5%/4.2%; MFRC: <1%) but miss 63–83% of true positives (MFTC mean FNR 63–65%; MFRC ~83%).
- Bias is not unidirectional: any-annotator over-labels; majority/two-vote under-labels. No rule is unbiased — only audited bias.
- Practical gain: training classifiers with calibrated posterior soft labels improves accuracy by 2–3% over hard labels, with no architectural changes.
- Why it matters: voting away disagreement destroys signal — Moral Foundations Theory itself predicts ideology-dependent disagreement, which is valuable information. The paper is a clean example of the proxy-objective trap: high recall under any-annotator hides false positives; high precision under majority hides most true positives. Per-annotator bias is multidimensional (unreliable on Care, accurate on Fairness), and the framework applies to any multi-annotator setting: content moderation, medical imaging, sentiment analysis.
- Limitations (acknowledged in the post): the posterior depends on prior choice (Beta-Binomial used); per-annotator confusion matrices need sufficient data; the "calibrated ground truth" was validated against eMFD expert labels, which may carry their own bias.
- Paper: https://arxiv.org/abs/2609.21992
- HTML: https://arxiv.org/html/2609.21992v1
- Code: https://github.com/maciejskorski/moral-entropy
Takeaway
The paper's posture is notable: rather than proposing yet another aggregation rule, it provides a framework to audit all rules, quantifying how far and in which direction each deviates from a calibrated posterior. As the post puts it: there is no unbiased rule — only audited bias. Preserving and modeling moral ambiguity beats voting it away.
Links: