Fair Representation Learning with Maximum Mean Discrepancy Distance Constraint (Student Abstract)

Alexandru Lopotenco, Ian Tong Pan, Jack Zhang, Guan Xiong QiaoAAAI 2024 · DOI

TL;DR

This paper introduces a novel framework for learning fair data representations using Maximum Mean Discrepancy (MMD) as a constraint. By penalizing the difference between distributions of data points belonging to different protected groups, their method aims to create representations that are both informative and unbiased, offering a generalized approach for various dimensionality reduction techniques.

What Problem Does This Paper Address?

In machine learning, especially with the growing emphasis on fairness and explainability, a significant challenge is ensuring that learned representations of data do not perpetuate or amplify existing societal biases. While techniques like PCA and t-SNE are powerful for dimensionality reduction, they often don't inherently consider fairness. This paper addresses the gap in generalized frameworks for fair and explainable representation learning, specifically focusing on how to remove information related to protected attributes (like gender) from these learned representations. The goal is to achieve fairness by ensuring that subsets of data with different protected attributes have similar underlying distributions in the learned representation space.

Methods

The core of the paper's contribution lies in its constrained optimization approach. They propose to minimize a standard loss function (e.g., reconstruction error for PCA, KL divergence for t-SNE) subject to a constraint on the Maximum Mean Discrepancy (MMD) between representations of data points belonging to different protected groups. MMD, a kernel-based method, is used to measure the distance between these distributions.

To solve this constrained optimization problem, they employ a Lagrangian method, effectively transforming it into an unconstrained problem with a penalty term. This penalty term is β * MMD(Y(0), Y(1))^2, where β is a Lagrangian coefficient. Since the problem is non-convex, they introduce an algorithm that uses a binary search to find an appropriate value for β, iteratively adjusting it and performing gradient descent to optimize the representation. An alternative approach is also suggested for cases where the primary loss function f is not differentiable or is difficult to work with, by substituting it with MMD of the initial representation.

Datasets

The paper uses a subset of 1000 samples from the Census Income dataset, publicly available from the UCI Machine Learning Repository. This dataset was chosen for its categorical features, which were one-hot encoded. Gender was used as the protected attribute for their experiments.

Key Insights and Ideas

  • MMD as a Fairness Metric: The paper highlights MMD's utility not just for measuring distribution differences but specifically for enforcing fairness by minimizing discrepancies between protected groups.
  • Generalized Framework: The proposed method is designed to be applicable to various dimensionality reduction techniques (like PCA, t-SNE, autoencoders) by abstracting the core loss function f.
  • Trade-off Management: The Lagrangian multiplier β and the binary search mechanism allow for controlling the trade-off between representation fidelity (minimizing f) and fairness (minimizing MMD).
  • Visualizing Fairness: The use of visualizations (scatter plots colored by protected attribute and another sensitive attribute like income) effectively demonstrates the impact of the fairness constraint on the learned representations.

Key Results

The paper presents qualitative results using t-SNE on the Census Income dataset.

  • Baseline t-SNE: Without fairness constraints, standard t-SNE clearly separates male and female data points, indicating an unintended bias.
  • Fair t-SNE (Algorithm 1): When applying their algorithm with fairness constraints (specifically, setting the MMD constraint δ to fractions of the original t-SNE MMD), the separation between genders becomes significantly less apparent.
  • Preservation of Structure: Notably, even with the fairness constraint, the income classes (low vs. high income) remained relatively separated, suggesting that the algorithm successfully preserved important data structure while achieving fairness.

Limitations and Future Work

  • Non-convexity: The authors acknowledge that the optimization problem is non-convex, which can lead to local optima. Their proposed algorithm attempts to mitigate this through iterative optimization and binary search for β.
  • Differentiability Assumption: The alternative approach relies on the differentiability of the primary loss function f. If f is not differentiable, a different substitution strategy might be needed.
  • Future Work: The authors encourage further experimentation with their framework on other models like autoencoders and PCA, and with different datasets and protected attributes. They also suggest exploring more complex fairness definitions.

Summary updated

BibTeX
@inproceedings{lopotenco2024fair,
  author = {Alexandru Lopotenco and Ian Tong Pan and Jack Zhang and Guan Xiong Qiao},
  title = {{Fair Representation Learning with Maximum Mean Discrepancy Distance Constraint (Student Abstract)}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2024},
  doi = {10.1609/aaai.v38i21.30476},
}
Figure