Mitigating Non-Linear Bias via Statistical Independence

This direction is split into other directions: non-linear bias detection. Its last version stays readable and citable here.

Version 1 of 166 papers93 conceptsfrozen 2026-10-04

Traditional algorithmic fairness relies on a dangerous assumption: that if a sensitive attribute (like race) and a model's prediction are uncorrelated, the model is fair. However, as Ruggero Ragonesi, Riccardo Volpi, and Jacopo Cavazza argue in Learning Unbiased Representations via Mutual Information Backpropagation, machine learning models are "notorious cheaters." They often bypass simple linear constraints by finding complex, non-linear proxies for protected information.

The emerging consensus across recent research is that true fairness requires moving beyond linear decorrelation toward statistical independence.

The Failure of Linear Decorrelation

Linear correlation (like Pearson’s $r$) only measures the degree to which two variables move together in a straight line. It is blind to higher-order dependencies. For example, if a sensitive attribute $S$ and a prediction $Y$ have a U-shaped relationship ($Y = S^2$), their linear correlation might be zero, yet $Y$ is entirely determined by $S$.

Vincent Grari, sylvain lamprier, and Marcin Detyniecki demonstrate in Fairness-Aware Neural Rényi Minimization for Continuous Features that models can appear fair under linear analysis while still exhibiting significant bias. This is particularly problematic in high-dimensional data like text or images, where Jian Huang notes in Fair Sufficient Representation Learning that biases are encoded in subtle, non-obvious patterns that standard linear methods fail to detect.

Reconstructing Sensitive Information via Proxies

Even if you explicitly remove a sensitive attribute, a model can often reconstruct it from other features. This "proxy discrimination" is rarely linear. Melanie Herschel and Nico Lässig point out in FALCC: Efficiently performing locally fair and accurate classifications that these intricate dependencies allow models to "leak" sensitive information back into the decision-making process.

This leakage is often intersectional. Senrong Xu and colleagues show in Fair Conformal Classification via Learning Representation-Based Groups that bias frequently arises from non-linear interactions between multiple features (e.g., the intersection of race, gender, and age) rather than single attributes in isolation.

Tools for Achieving True Independence

To combat non-linear bias, researchers are turning to three primary mathematical frameworks that capture the "full" relationship between variables:

1. Maximal Correlation (HGR)

The Hirschfeld-Gebelein-Rényi (HGR) coefficient is a "maximal" correlation measure. It searches for the most complex non-linear transformations of two variables that could possibly correlate. Joshua Lee and Gregory Wornell propose using HGR as a regularizer in A Maximal Correlation Approach to Imposing Fairness in Machine Learning. If the HGR coefficient is zero, the variables are truly independent.

2. Hilbert-Schmidt Independence Criterion (HSIC)

HSIC is a kernel-based method that measures independence by looking at similarities in high-dimensional "feature spaces." Alex Gittens and Kevin Kim apply this to tensor decompositions in Learning Fair Canonical Polyadical Decompositions using a Kernel Independence Criterion, arguing that standard orthogonality (linear) is insufficient to stop downstream classifiers from recovering protected information.

3. Mutual Information (MI)

MI measures the "number of bits" shared between two variables. Chen Zhao and Le Wu use an MI-based objective in Fair Representation Learning for Recommendation: A Mutual Information Perspective to minimize the information shared with sensitive attributes while maximizing task-relevant information.

Scrubbing the Latent Space

The goal of these methods is often "concept erasure" or "scrubbing"—transforming data representations so that no model, no matter how complex, can extract the sensitive attribute.

Felix Störck, Fabian Hinder, and Barbara Hammer extend this to kernel methods in Extending Fair Null-Space Projections for Continuous Attributes to Kernel Methods, providing a way to "scrub" continuous information like age from non-linear models. Similarly, Ramin Akbari and Vishnu Boddeti introduce Obliviator, which uses an iterative kernel approach to protect against sophisticated non-linear "attacks" that try to recover erased concepts.

By shifting the focus from correlation to independence, this research direction ensures that fairness isn't just a surface-level statistical property, but a fundamental characteristic of the model's internal logic.

Papers in this version

Showing 60 of 66 papers.

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_mitigating_non_linear_bias_via_statistic_d7c78ab2,
  title = {Mitigating Non-Linear Bias via Statistical Independence},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/mitigating-non-linear-bias-via-statistical-independence/rdr_d7c78ab2686541c5b1fba727016b18ca},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_mitigating_non_linear_bias_via_statistic_d7c78ab2_v1,
  title = {Mitigating Non-Linear Bias via Statistical Independence},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/mitigating-non-linear-bias-via-statistical-independence/rdr_d7c78ab2686541c5b1fba727016b18ca/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}