Metric Fairness and the Challenge of Individual Similarity

You are reading version 1 of this direction, frozen 2026-10-04. The latest version is version 2.

Version 1 of 2104 papers131 conceptsfrozen 2026-10-04

Most algorithmic fairness research focuses on group fairness: ensuring that statistical outcomes (like loan approval rates) are roughly equal across broad demographic categories like race or gender. However, group fairness can be a blunt instrument. It is possible to satisfy group-level statistics while still discriminating against specific individuals or subgroups—a phenomenon Sherif Botros and Vanio Markov identify as a critical flaw in current lending regulations in Black Loans Matter.

The alternative is individual fairness, a principle famously articulated by Moritz Hardt, Omer Reingold, Richard Zemel, and colleagues in Fairness Through Awareness: "Similar individuals should be treated similarly."

The Lipschitz Condition for Fairness

To turn this intuitive principle into a mathematical constraint, researchers use a similarity metric. If we have two individuals, $x$ and $y$, and a distance function $d(x, y)$ that measures how similar they are for a specific task, an individually fair algorithm $M$ must satisfy a Lipschitz condition:

$$d_{out}(M(x), M(y)) \leq d_{in}(x, y)$$

In plain English: the difference in the outcomes (the "distance" in the output space) cannot be greater than the difference in the individuals themselves (the "distance" in the input space). If two people are nearly identical for the purposes of a loan, the algorithm is mathematically forbidden from giving them significantly different interest rates.

The Metric Problem: Who Defines Similarity?

The mathematical elegance of individual fairness masks a profound practical difficulty: where does the metric $d(x, y)$ come from?

As Richard A. Berk and Hoda Heidari argue in A Convex Framework for Fair Regression, selecting a metric is not a technical task but a subjective policy challenge. Standard mathematical proxies like Euclidean distance often fail to capture the social or legal nuances of fairness. For example, in a university admissions context, a model might be "consistent" but still unfair if it ignores the fact that two applicants with different test scores might be "clinically similar" once their socioeconomic backgrounds are considered, a concept explored by Yeon Mi Hwang in the EquiLense framework.

Learning the Metric from Humans and Data

Because defining a metric upfront is so difficult, a major branch of research focuses on learning the metric from other sources:

  1. Human Judgment: Hanchen Wang and Preethi Lahoti show in their empirical study that we can train models on human comparisons to derive fairness metrics that outperform standard mathematical baselines. Similarly, Florian E. Dorner and Naman Goel use human-in-the-loop frameworks to generate context-aware sentence pairs for NLP fairness.
  2. Weak Supervision: In Online Learning with an Unknown Fairness Metric, Stephen Gillen and Aaron Roth propose an algorithm that learns the metric on the fly by receiving feedback from a regulator who points out violations without needing to define the entire metric in advance.
  3. Data Representations: Frameworks like iFair, developed by Preethi Lahoti and Gerhard Weikum, attempt to learn low-rank representations of data that "hide" sensitive attributes while preserving the features that define individual similarity.

Fairness as Robustness

A powerful intuition emerging in the field is the link between individual fairness and adversarial robustness. In robustness, we want a model's prediction to stay the same even if an attacker slightly perturbs the input. In fairness, we want the prediction to stay the same even if we "perturb" an individual's sensitive attributes.

Mikhail Yurochkin and Yuekai Sun formalized this in SenSR (Sensitive Subspace Robustness). Instead of requiring a full metric, SenSR identifies "sensitive directions" in the data (like gender-coded words in text) and uses distributionally robust optimization to ensure the model is invariant to changes along those axes. This "flips" the problem: rather than defining who is similar, we define what differences should not matter.

Verifying and Auditing Fairness

As these models move into production, the focus shifts to verification. Yasser Shoukry introduced CertiFair, a framework that can mathematically prove whether a neural network is globally fair across its entire domain or provide a specific counterexample where the Lipschitz condition is violated.

Ultimately, the research direction suggests that while the mathematical tools for individual fairness are becoming highly sophisticated, the "ground truth" of fairness remains a human-centric, domain-specific decision. As Matt Fredrikson and Samuel Yeom note in Individual Fairness Revisited, technical frameworks can provide guarantees, but they cannot resolve the subjective societal challenge of deciding which individuals truly count as "similar."

Papers in this version

Showing 60 of 104 papers.

History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 114 papers
  2. Version 1 · 2026-10-04 · 104 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_similarity_metric_specification_c704e16c,
  title = {similarity metric specification},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/similarity-metric-specification/rdr_c704e16c59984dc2964e362992f6b945},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_metric_fairness_and_the_challenge_of_ind_c704e16c_v1,
  title = {Metric Fairness and the Challenge of Individual Similarity},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/metric-fairness-and-the-challenge-of-individual-similarity/rdr_c704e16c59984dc2964e362992f6b945/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}