When evaluating whether an algorithmic decision-making system is "fair," researchers and practitioners generally look through one of two lenses. The first is group fairness, which demands that different demographic groups (such as those defined by race, gender, or age) receive equitable aggregate outcomes. The second is individual fairness, which requires that similar individuals be treated similarly, regardless of their group membership.
While both objectives are socially and ethically desirable, they are often fundamentally at odds. As Richard S. Zemel et al. observed in their foundational paper Learning Fair Representations, a structural tension exists between population-level demographic parity and individual-level consistency. When an algorithm is forced to satisfy macro-level group quotas, it must frequently distort how it evaluates micro-level individual merits.
The Mechanics of the Conflict
To understand why this conflict arises, consider a predictive model used for bank loans or school admissions. Suppose the underlying historical data contains disparities, leading to different baseline qualification distributions between two demographic groups, Group A and Group B.
To achieve group fairness—such as demographic parity—the algorithm must ensure that the selection rate for both groups is identical. To do this, the model cannot apply a single, uniform decision threshold across the entire population. Instead, it must effectively lower the acceptance threshold for the underrepresented group or raise it for the majority group.
This threshold-shifting introduces an immediate violation of individual fairness. If the model evaluates two applicants who possess identical qualifications but belong to different groups, the group-level adjustment will force the model to give them different outcomes. As Amanda Bower et al. demonstrate in Individually Fair Rankings, enforcing population-level demographic quotas inherently requires treating highly similar, qualified individuals differently based strictly on their protected attributes.
Without Group Constraints:
[Unified Threshold] ───> Evaluates purely on individual features. Similar people get similar outcomes.
With Group Constraints (Demographic Parity):
[Group A Threshold] ───> Raised to restrict acceptances.
[Group B Threshold] ───> Lowered to boost acceptances.
Result: Two identical individuals near the threshold receive opposite decisions.
The Hidden Cost of Within-Group Unfairness
The tension does not just manifest between members of different groups; it can paradoxically worsen fairness within the very communities it aims to protect.
When a model is constrained to hit a specific aggregate group metric, it often creates a compressed or distorted distribution of utility. For instance, Aniko Hannak et al. revealed in Fair Machine Learning Through Post-processing: The Case of Predictive Parity that to achieve predictive parity across groups, an optimal decision-maker may actually be forced to reject the most qualified individuals within a specific group. By trying to balance aggregate statistics, the model introduces severe within-group unfairness.
This phenomenon can be precisely quantified using economic principles. By adapting inequality indices from economics, Till Speicher et al. proved in A Unified Approach to Quantifying Algorithmic Unfairness: Measuring Individual &Group Unfairness via Inequality Indices that minimizing the "between-group" component of unfairness frequently causes a corresponding spike in "within-group" unfairness. The aggregate group metric acts as a blunt instrument, masking deep disparities and arbitrary treatment at the individual level.
How Mitigation Strategies Degrade Individual Equity
Engineers typically attempt to mitigate algorithmic bias at three distinct stages: pre-processing (modifying the training data), in-processing (modifying the objective function during training), or post-processing (tweaking the final outputs). Unfortunately, when these interventions are guided solely by group metrics, they often degrade individual consistency in predictable ways.
Data Pre-processing and Feature Distortion
One common approach is to reweight or transform the training dataset so that demographic groups appear balanced before a model is trained. However, Mikhail Yurochkin et al. showed in Auditing ML Models for Individual Bias and Unfairness that these interventions cause the model to rely more heavily on protected attributes to balance group averages, thereby increasing individual unfairness. Similarly, the authors of Optimized Pre-Processing for Discrimination Prevention noted that transforming data to eliminate group discrimination can map similar individuals to significantly different representations unless a strict distortion budget is maintained.
The Illusion of Stochasticity
Another mathematical workaround to satisfy group metrics like "Equalized Odds" is to introduce randomness into predictions. While Zeyu Tang et al. prove in Attainability and Optimality: The Equalized Odds Fairness Revisited that stochasticity makes group fairness mathematically attainable for deterministic models, it introduces a severe practical vulnerability: it destroys individual consistency. Two identical individuals can receive completely different outcomes purely due to a randomized coin flip, undermining the core tenet of individual equity.
Auditing and Structural Failures
Statistical audits by Songkai Xue et al. in Statistical inference for individual fairness confirm that optimizing strictly for group-level metrics paradoxically increases individual fairness violations. Because group metrics are aggregate averages, they are blind to local structural changes. In graph-based learning, for example, Charlotte Laclau et al. found in All of the Fairness for Edge Prediction with Optimal Transport that aggressive group-level adjustments distort the local structural information of a graph, destroying the relative positioning of similar nodes.
Moving Toward Joint Optimization
Recognizing that group and individual fairness are in inherent tension, recent research focuses on frameworks that explicitly balance both objectives rather than treating them as mutually exclusive.
To prevent data cleaning from distorting individual profiles, Flavio Calmon et al. introduced optimization frameworks that enforce a strict individual distortion budget while simultaneously reducing group-level discrimination. For black-box models where internal mechanics cannot be altered, Karthikeyan Natesan Ramamurthy et al. proposed post-processing frameworks like Individual+Group Debiasing (IGD) in Bias Mitigation Post-processing for Individual and Group Fairness. IGD explicitly flags individuals likely to be victims of group-level quota adjustments and prioritizes them for corrective, consistent treatment.
Ultimately, as Yijun Bian et al. suggest in Increasing Fairness via Combination with Learning Guarantees, fairness cannot be measured along a single dimension. By framing fairness through unified metrics that encompass both individual and group biases simultaneously, machine learning practitioners can move away from blunt demographic quotas and toward systems that respect both population equity and individual merit.
Key Foundational Work
Learning Fair Representations (2013) — contributed 1 concept elements to this direction.
Optimized Pre-Processing for Discrimination Prevention (2017) — contributed 1 concept elements to this direction.