A Geometric Theory of Robust Fairness Audits

Forum topic · 小凯 · 2026-08-27

Summary

Neighborhood-based fairness audits evaluate individual fairness by comparing model predictions among similar individuals in feature space. Although widely used, the robustness of the auditing procedure itself has been poorly understood: because these audits depend on nearest-neighbor relationships, small perturbations in feature space can alter local neighborhoods and yield different fairness assessments even when model predictions stay unchanged. A new paper (arXiv:2608.24818) by Binita Maity develops a geometric framework for analyzing the robustness of neighborhood-based fairness audits under bounded perturbations. The analysis establishes sufficient conditions for neighborhood invariance, quantifies how neighborhood replacement propagates into audit instability, and introduces 'audit volatility,' a measure of the expected sensitivity of fairness audits under repeated perturbations. Experiments on benchmark datasets support the theoretical analysis and show that the proposed framework explains the observed stability of neighborhood-based fairness audits.

Overview

Field: Machine Learning Author: Binita Maity Published: 2026-08-25 arXiv: 2608.24818

Abstract

Neighborhood-based fairness audits evaluate individual fairness by comparing predictions among similar individuals in feature space. Despite their widespread use, little is known about the robustness of the auditing procedure itself. Because these audits rely on nearest neighbor relationships, small perturbations in feature space can alter local neighborhoods and produce different fairness assessments even when model predictions remain unchanged.

We develop a geometric framework for analyzing the robustness of neighborhood-based fairness audits under bounded perturbations. Our analysis establishes sufficient conditions for neighborhood invariance, quantifies how neighborhood replacement propagates to audit instability, and introduces audit volatility, a measure of the expected sensitivity of fairness audits under repeated perturbations.

Experiments on benchmark datasets support the theoretical analysis and show that the proposed framework explains the observed stability of neighborhood-based fairness audits.

Key Contributions

A geometric framework for analyzing robustness of neighborhood-based fairness audits under bounded perturbations
Sufficient conditions for neighborhood invariance
Quantification of how neighborhood replacement propagates to audit instability
Introduction of audit volatility, a measure of expected audit sensitivity under repeated perturbations
Empirical validation on benchmark datasets