Traditional algorithmic fairness often focuses on coarse, predefined demographic groups, such as race or gender. However, real-world bias is frequently intersectional and granular. If a model is optimized to be fair only to broad categories, it can still severely disadvantage specific combinations of attributes—a phenomenon known as "fairness gerrymandering."
To prevent this, we ideally want to protect every possible subpopulation, even without explicit demographic labels. Yet, as we slice the population into finer and more granular segments, we run into a fundamental mathematical and statistical barrier. Recent research reveals that we cannot protect arbitrarily small groups without destroying the model's utility. Instead, robust fairness frameworks must rely on subpopulation size thresholding—setting a strict lower bound on the size of a subgroup before enforcing fairness constraints.
The Mathematical Floor: Why Small Subgroups Destroy Utility
Why can we not simply demand fairness for every single subpopulation, down to groups of just a few individuals? In Blind Pareto Fairness and Subgroup Robustness, Martin Andres Bertran et al. prove that there is a theoretical lower bound on subgroup size, termed a "fairness granularity floor." Crossing below this critical partition size mathematically forces the model to degrade into a trivial, uniform random classifier.
To understand the intuition, imagine a subgroup so small or sparse that the available features contain no predictive information for it, or its labels are highly contradictory. The model's error rate on this tiny group will naturally be high. If a strict fairness constraint mandates that the model achieve equal error rates across all subgroups, the optimization algorithm faces a dilemma. It cannot magically extract predictive signals where none exist to improve the tiny group. Therefore, the only way to satisfy the mathematical constraint of equal performance is to intentionally destroy the model's performance on the majority population.
When fairness constraints are forced upon subgroups below the critical size threshold, group equity and predictive utility become mutually exclusive. The optimization satisfies fairness by making the model equally uninformative for everyone.
The Statistical Hazard: Overfitting to Sample Noise
Even if a model avoids total collapse into a random classifier, protecting minuscule subgroups introduces severe statistical instability. As explored in Doubly-Regressing Approach for Subgroup Fairness by Kunwoong Kim et al., empirical fairness metrics calculated on tiny samples are highly volatile and fail to generalize to the broader population.
Consider an intersectional subgroup in a training dataset that contains only three individuals. Any measurement of "fairness" or error rate on this group is dominated by sampling noise rather than a systemic trend. If the model alters its decision boundaries to satisfy a fairness constraint for these three individuals, it is essentially overfitting to noise. When deployed on unseen data, this constraint will likely fail to protect the actual demographic group it was intended for.
To solve this, the authors propose filtering for "active subgroups," enforcing fairness constraints exclusively on attribute intersections that exceed a specific population threshold. This threshold acts as a statistical shield, ensuring that the model only calibrates its fairness constraints on data subsets large enough to yield stable, reliable estimates.
Operationalizing Thresholds via Distributionally Robust Optimization
How can practitioners implement this size threshold in practice, especially when demographic labels are missing or unknown during training? A powerful framework for achieving this is Distributionally Robust Optimization (DRO).
In Distributionally Robust Survival Analysis: A Novel Fairness Loss Without Demographics, Shu Hu et al. leverage DRO to protect unknown subgroups. Instead of optimizing for the average performance, the framework minimizes the worst-case error across all possible subpopulations that meet a minimum size threshold, defined as a fraction $\alpha$ of the total population.
The parameter $\alpha$ acts as a tunable control knob for the fairness-utility trade-off:
- Small $\alpha$: The model attempts to protect highly specific, smaller minority groups. This extends protection deeper into intersectional identities but risks approaching the fairness granularity floor, which increases the trade-off in overall model accuracy.
- Large $\alpha$: The model only optimizes for the worst-case performance of large, statistically robust population segments, preserving higher aggregate predictive power at the expense of ignoring niche minority groups.
By treating the subpopulation size as a continuous hyperparameter, practitioners can map out performance-disparity curves. As shown in subsequent work by Shu Hu et al. in Fairness in Survival Analysis with Distributionally Robust Optimization, these curves allow designers to visualize exactly how much global utility is conceded as the granularity floor is lowered, enabling a principled, data-driven choice for where the subpopulation threshold should stand.
Key Foundational Work
Blind Pareto Fairness and Subgroup Robustness (2021) — contributed 8 concept elements to this direction.
Distributionally Robust Survival Analysis: A Novel Fairness Loss Without Demographics (2022) — contributed 2 concept elements to this direction.