Fairness-Aware Neural Rényi Minimization for Continuous Features

Vincent Grari, Sylvain Lamprier, Marcin Detyniecki

Introduction

Fairness in machine learning has traditionally focused on discrete sensitive attributes, such as binary gender or race categories. However, many critical real-world variables—like age, income, or credit scores—are continuous. Standard fairness metrics often rely on linear correlations, which fail to capture complex, non-linear dependencies. For instance, a model might show zero linear correlation with age but still exhibit a "U-shaped" bias, treating both the very young and very old individuals differently.

This paper introduces Fair HGR NN, a novel adversarial neural network framework designed to decouple model predictions from continuous sensitive attributes. By directly minimizing the Hirschfeld-Gebelein-Rényi (HGR) maximal correlation coefficient, the authors ensure independence even under highly non-linear relationships.

Methodology: Neural HGR Minimization

The core of the proposed "in-processing" mitigation strategy is the HGR coefficient. Unlike Pearson’s rho, HGR searches for the maximum possible correlation between any non-linear transformations of two variables. Independence is achieved only if the HGR is zero.

The authors implement this via an adversarial minimax game:

  1. Predictor ($h_\phi$): A network trained to minimize prediction error (e.g., MSE) on the target $Y$.
  2. Adversary ($f_{\theta_f}, g_{\theta_g}$): Two networks that collaborate to find non-linear transformations maximizing the correlation between the predictor's output and the sensitive attribute $S$.

A hyperparameter $\lambda$ controls the trade-off between accuracy and fairness. As $\lambda$ increases, the model prioritizes reducing the HGR coefficient at the cost of a slight increase in prediction error.

The trade-off between prediction error (MSE) and fairness (HGR) as controlled by the hyperparameter λ.

Experimental Evaluation

The framework was evaluated on synthetic data and three real-world datasets: US Census demographics, French motor insurance claims, and UCI crime rates.

Robustness to Noise

A key advantage of the neural HGR estimator over traditional methods like Kernel Density Estimation (KDE) is its robustness. In scenarios with significant noise or "unsteady" relationships, the neural approach maintains a more stable and accurate estimation of the underlying dependency.

Comparison of HGR estimation robustness against noise for different non-linear functions (sinusoidal, step, cubic, quadratic). The neural approach (HGR_NN) consistently outperforms KDE-based methods.

Key Results

  • Non-Linear Bias Removal: On synthetic data, Fair HGR NN successfully eliminated quadratic biases that standard neural networks ignored.
  • State-of-the-Art Performance: On the US Census and Crime datasets, the method achieved superior fairness scores (HGR, FairQuant) while maintaining competitive MSE compared to baselines like MINE or KDE-based learners.
  • Flexibility: The approach supports both Demographic Parity and Equalized Residuals objectives.

Discussion and Future Work

The research highlights that "fairness through unawareness" is insufficient, as proxy features often harbor non-linear traces of sensitive data. HGR serves as a rigorous "stress test" for auditing such biases.

However, challenges remain. On smaller datasets, HGR estimation can be volatile. Additionally, like most adversarial frameworks, training requires precise tuning of learning rates to prevent oscillations. Future work may explore extending this framework to multi-dimensional sensitive attributes or different model architectures.

Reference: Grari, V., Lamprier, S., & Detyniecki, M. (2020). Fairness-Aware Neural Rényi Minimization for Continuous Features. IJCAI-20.

Summary updated

Abstract

The past few years have seen a dramatic rise of academic and societal interest in fair machine learning. While plenty of fair algorithms have been proposed recently to tackle this challenge for discrete variables, only a few ideas exist for continuous ones. The objective in this paper is to ensure some independence level between the outputs of regression models and any given continuous sensitive variables. For this purpose, we use the Hirschfeld-Gebelein-Rényi (HGR) maximal correlation coefficient as a fairness metric. We propose to minimize the HGR coefficient directly with an adversarial neural network architecture. The idea is to predict the output Y while minimizing the ability of an adversarial neural network to find the estimated transformations which are required to predict the HGR coefficient. We empirically assess and compare our approach and demonstrate significant improvements on previously presented work in the field.
BibTeX
@inproceedings{grari2020fairness,
  author = {Vincent Grari and Sylvain Lamprier and Marcin Detyniecki},
  title = {{Fairness-Aware Neural Rényi Minimization for Continuous Features}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2020},
  doi = {10.24963/ijcai.2020/313},
}
Figure