Fairness in Link Prediction Beyond Demographic Parity: Reproducibility Study on NDKL and MORAL

Forum topic · 小凯 · 2026-08-12

Summary

This paper reproduces and extends a study on fairness in ranked link prediction, arguing that demographic parity (Δ_DP) is insufficient because it ignores rank position and can show aggregate parity even when subgroup-pair links are systematically ranked lower. The authors validate that the rank-aware Normalized Discounted KL-divergence (NDKL) detects such exposure disparities, and they reproduce MORAL, a post-processing method that improves exposure-based fairness while maintaining competitive utility. Beyond reproduction, the team evaluates robustness over synthetic homophily settings, categorical sensitive attributes, and additional fairness/utility metrics, including subgroup-pair-adaptive AWRF. Findings indicate that exposure-based metrics reveal bias hidden by Δ_DP, and MORAL reduces this bias with minimal utility loss across diverse datasets. The corrected reproducible implementation is released on GitHub.

Paper Overview

Research Area: Machine Learning (ML)
Authors: Valentijn Oldenburg, Floris de Kam, Stef de Wildt
Published: 2026-08-12
arXiv: 2508.05138

Abstract

In fair ranked link prediction, demographic parity (Δ_DP) is a common fairness metric. Yet, Mattos et al. (2025) argue that it fails to detect exposure bias because it ignores where links appear in the ranking. In this study, we reproduce this claim by showing that Δ_DP can indicate aggregate parity even when some subgroup-pair links are systematically ranked lower than others. The proposed rank-aware Normalized Discounted KL-divergence (NDKL), however, does detect such disparities. We also reproduce the effectiveness of MORAL, a post-processing method that improves exposure-based fairness while maintaining competitive utility. Beyond reproduction, we assess robustness using synthetic homophily settings, categorical sensitive attributes, and additional fairness and utility metrics, including subgroup-pair-adaptive Attention-Weighted Rank Fairness (AWRF). Across these experiments, exposure-based metrics reveal bias hidden by Δ_DP, and MORAL reduces such bias with minimal utility loss on diverse datasets. A corrected reproducible implementation is released at https://github.com/Floris93100/reproducing-MORAL.

Key Points

Research Question: Does demographic parity (Δ_DP) adequately measure fairness in ranked link prediction, or does it mask exposure-based bias by ignoring rank position?
Methodology: Reproduction of Mattos et al. (2025), complemented by synthetic homophily experiments, categorical sensitive attributes, and additional metrics.
Main Findings:
Δ_DP can show aggregate parity even when certain subgroup-pair links are systematically ranked lower, confirming the original critique.
The rank-aware Normalized Discounted KL-divergence (NDKL) successfully detects such disparities.
MORAL, a post-processing method, improves exposure-based fairness while preserving competitive utility.
Attention-Weighted Rank Fairness (AWRF) provides a subgroup-pair-adaptive view of exposure fairness.
Robustness: Results hold across diverse datasets and synthetic settings, with minimal utility cost when applying MORAL.
Artifacts: Corrected reproducible code published at https://github.com/Floris93100/reproducing-MORAL.

Resources

Paper: https://arxiv.org/abs/2508.05138
Code: https://github.com/Floris93100/reproducing-MORAL