Differentiable Surrogates for Algorithmic Fairness

90 concepts70 papers1 proposal

Standard machine learning models learn by sliding down a smooth mathematical landscape via gradient descent. When a model's weights change slightly, its predictive loss changes slightly, providing a clear signal for backpropagation. However, traditional group fairness metrics—such as demographic parity, equalized odds, or disparate mistreatment—are fundamentally discrete. They rely on counting operations, indicator functions, or hard decision thresholds (e.g., whether an individual's predicted score falls above or below $0.5$).

This creates a severe mathematical friction. A step function has a gradient of zero almost everywhere and is discontinuous at the threshold. If a model's prediction for an individual shifts from $0.1$ to $0.49$, a threshold-based fairness metric registers zero change, offering no gradient signal to guide the optimization loop. To resolve this, a major research direction focuses on differentiable fairness optimization: transforming discrete, non-differentiable fairness criteria into smooth, continuous mathematical surrogates that can be optimized directly alongside accuracy during training.

Relaxing Indicators to Convex Proxies

One of the earliest and most intuitive ways to smooth the fairness landscape is to replace hard indicator functions with continuous algebraic proxies. Instead of evaluating fairness based on final binary classifications, these methods look at the continuous margins or distances from the model's decision boundary.

As shown by Muhammad Bilal Zafar et al. in Fairness Beyond Disparate Treatment & Disparate Impact: Learning Classification without Disparate Mistreatment, rather than directly equalizing discrete misclassification rates across groups, we can measure the covariance between a user's sensitive attribute and their signed distance from the decision boundary. Because the distance to the boundary is a continuous function of the model's weights, its covariance is fully differentiable, allowing the integration of fairness requirements directly into standard loss minimization frameworks. Similarly, Blake Woodworth et al. in Learning Non-Discriminatory Predictors replace strict conditional independence requirements with a second-moment relaxation ("equalized correlations"), matching the covariances between predictions and protected attributes to make the optimization problem convex and computationally tractable.

Other approaches substitute indicator functions with smooth upper bounds, mirroring how support vector machines use hinge loss as a proxy for 0/1 classification error. For instance, Chris Russell et al. in When Worlds Collide: Integrating Different Counterfactual Assumptions in Fairness utilize convex hinge-loss relaxations to penalize counterfactual disparities.

However, unconstrained linear or non-smooth relaxations can sometimes cause flexible classifiers like deep neural networks to destabilize or collapse during training. To address this, Harry Bendekgey in Scalable and Stable Surrogates for Flexible Classifiers with Fairness Constraints demonstrates that using bounded and convex surrogates—such as smooth cross-entropy approximations—provides a stable upper bound on fairness violations that prevents the constraint term from destroying the global loss landscape.

Smoothing Distributions with Kernel Density Estimation

When fairness metrics depend on the entire distribution of outputs rather than just a boundary margin, simple covariance proxies fall short. For example, evaluating demographic parity across all possible classification thresholds requires comparing the cumulative distribution functions of different groups.

To make these distribution-level properties differentiable, researchers employ Kernel Density Estimation (KDE). Instead of treating predictions as hard binary outcomes, the authors of A Fair Classifier Using Kernel Density Estimation use KDE to approximate the continuous probability density function of the classifier's outputs. By placing a smooth, continuous kernel (such as a Gaussian function) over each individual prediction, the discrete counting steps are smoothed out into a continuous curve.

Discrete Counting (Non-Differentiable):
Subgroup A: [████████████░░░░░░] -> Hard Threshold -> Step Function (Gradient = 0)

Kernel Density Estimation (Differentiable):
Subgroup A:  ───/\_/\_/\───────> Smooth Probability Curve (Gradient Flow Enabled)

This transformation allows the model to compute how a tiny adjustment to its weights shifts the entire probability mass of a demographic group. As a result, researchers such as Xiaotian Han et al. in Retiring ΔDP: New Distribution-Level Metrics for Demographic Parity can optimize threshold-invariant, distribution-level fairness metrics directly within the training loop via standard backpropagation.

Distributional Distance and Divergence Penalties

When the goal is to ensure that a model's predictions or internal representations are entirely independent of a sensitive attribute, the optimization problem can be framed as minimizing the statistical divergence between group distributions.

For continuous sensitive attributes (like age or income) where discrete group buckets do not exist, Jeremie Mary et al. in Fairness-Aware Learning for Continuous Attributes and Treatments leverage the Rényi maximum correlation coefficient, upper-bounding it with a differentiable $\chi^2$-divergence penalty to enable end-to-end mini-batch stochastic gradient descent.

Alternatively, optimal transport metrics can be used to align prediction distributions. Laurent Risser et al. in Tackling Algorithmic Bias in Neural-Network Classifiers using Wasserstein-2 Regularization apply Gâteaux derivatives to calculate the gradient of a global Wasserstein-2 distance penalty with respect to individual predictions. This mathematical framework allows standard gradient descent to enforce strict distributional alignment across subgroups without relying on unstable adversarial predictors. In generative modeling contexts where demographic labels are entirely missing, Soobin Um et al. in A Fair Generative Model Using LeCam Divergence utilize variational optimization to convert robust statistical divergence measures (LeCam divergence) into differentiable loss functions using a small, balanced reference dataset.

Why It Works: In-Processing Integration

By transforming non-differentiable fairness constraints into smooth, continuous loss terms, this research direction moves fairness from an afterthought (post-hoc threshold adjustments) to a core component of the optimization process (in-processing).

This unified view is epitomized by modern software frameworks like fairret, developed by Maarten Buyl et al. in fairret: a Framework for Differentiable Fairness Regularization Terms. The framework maps a broad class of linear-fractional group statistics directly onto differentiable regularization terms. Because these surrogates are fully compatible with automatic differentiation, the model's internal representations are forced to internalize equity as a native objective alongside accuracy, achieving a more robust and stable balance between utility and fairness.

Key Foundational Work

Fairness Beyond Disparate Treatment & Disparate Impact: Learning Classification without Disparate Mistreatment (2017) — contributed 2 concept elements to this direction.
Beyond Parity: Fairness Objectives for Collaborative Filtering (2017) — contributed 1 concept elements to this direction.
When Worlds Collide: Integrating Different Counterfactual Assumptions in Fairness (2017) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_differentiable_fairness_optimization_8c36cf3d,
  title = {differentiable fairness optimization},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/differentiable-fairness-optimization/rdr_8c36cf3db1c145f6b6aabf4f0ba13b9b},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_differentiable_fairness_optimization_8c36cf3d_v1,
  title = {differentiable fairness optimization},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/differentiable-fairness-optimization/rdr_8c36cf3db1c145f6b6aabf4f0ba13b9b/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake