FADES: Fair Disentanglement with Sensitive Relevance

Taeuk Jang, Xiaoqian WangCVPR 2024 · DOI

Introduction

In the quest for "fair" AI, researchers often attempt to learn representations that are independent of sensitive attributes, such as race or gender. Most existing methods aim to "disentangle" latent variables into two perfectly separate categories: one for the target task (e.g., predicting if someone is smiling) and one for the sensitive attribute. However, this approach often fails in the real world where datasets are inherently biased and target labels are frequently correlated with sensitive attributes.

The Independence Contradiction

The authors of FADES (Fair DisEntanglement with Sensitive relevance) argue that forcing perfect independence between target and sensitive representations is a mathematical trap. For instance, in a specific dataset, a "bald" attribute might be statistically more common in men. If a model is forced to make these representations independent while maintaining high accuracy, the objectives contradict each other. This leads to unstable training, degraded performance, and "unrealistic counterfactuals"—such as a generative model creating an image of a girl with a mustache because it fails to understand how those features correlate.

The FADES Framework

FADES moves away from the "perfectly independent buckets" approach by explicitly modeling the overlap between variables. It utilizes a Variational Autoencoder (VAE) architecture that splits the latent space into four distinct parts:

  1. $z_Y$ (Target code): Features unique to the target label.
  2. $z_A$ (Sensitive code): Features unique to the sensitive attribute.
  3. $z_R$ (Sensitive relevant code): The core innovation, which explicitly captures the information that overlaps between the target and the sensitive attribute.
  4. $z_X$ (Background code): Irrelevant background features.
Comparison of learning paradigms: (a) Invariant learning, (b) Disentanglement learning, (c) Correlation-aware learning, and (d) the proposed FADES approach.

To implement this, FADES leverages Conditional Mutual Information (CMI). Instead of forcing $z_Y$ and $z_A$ to be independent, it enforces that the predictions of the target and sensitive attributes are independent given the shared information in $z_R$. Additionally, an Information Bottleneck regularization term prevents $z_R$ from absorbing excessive information, forcing it to capture only the correlated data necessary to satisfy fairness constraints.

Experimental Results

The authors evaluated FADES on a diverse mix of benchmarks, including CelebA, UTKFace, and tabular data like the Adult Income dataset.

  • Classification Performance: On the CelebA dataset, FADES achieved a high accuracy of 0.918 while maintaining a low Equalized Odds (EOD) score of 0.032, outperforming specialized models like FFVAE and GVAE.
  • Robustness to Bias: In experiments with C-MNIST (a color-biased digit dataset), FADES significantly outperformed others in recovering original digits (93.46% accuracy), proving its ability to "see through" spurious correlations.
  • Generative Quality: Image generation tests demonstrated that FADES maintains better background and identity consistency when interpolating sensitive attributes compared to previous methods.

Conclusion

By acknowledging and modeling the inherent correlations in biased data, FADES provides a more robust and realistic framework for fair machine learning. While the model requires careful hyperparameter tuning, its ability to handle "double agent" features—those carrying information about both the task and the sensitive attribute—marks a significant step forward in building AI systems that are both accurate and equitable.

Summary updated

BibTeX
@inproceedings{jang2024fades,
  author = {Taeuk Jang and Xiaoqian Wang},
  title = {{FADES: Fair Disentanglement with Sensitive Relevance}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2024},
  doi = {10.1109/cvpr52733.2024.01147},
}
Figure