Standard machine learning models are trained using Empirical Risk Minimization (ERM), an objective that minimizes the expected loss across a dataset. Mathematically, ERM treats every data point with equal weight, optimizing for the global mean performance.
While this strategy yields high aggregate accuracy, it introduces a systemic optimization bias: the model acts as a majority-rule system. If a model can achieve 99% overall accuracy by perfectly fitting a dominant majority pattern while completely failing on a 1% minority subgroup, an ERM objective considers this a massive success.
As Raman demonstrates in Centering the Margins, standard models function as "normalcy engines" that inherently disadvantage data points deviating from the statistical norm. This average-case bias masks severe performance deficits within minority subgroups, rare classes, and atypical edge cases.
Why Models Take Shortcuts
When an algorithm optimizes solely for average loss, it naturally falls into what Gengyu Lin et al. describe in FairSTG as a "lazy learning mode." Instead of learning robust, causal features that generalize across all examples, the model exploits high-frequency statistical shortcuts—or spurious correlations—present in the majority of the data.
As explored by Piratla et al. in Focus on the Common Good, a model might successfully classify a bird by associating it with a beach background simply because most bird photos in the training set happen to be taken at the coast. This shortcut minimizes average loss effectively, but it fails catastrophically when the model encounters "bias-conflicting" examples, such as a bird in a forest or an object in an unconventional context (Zheng et al. in Learning Robust Classifiers with Self-Guided Spurious Correlation Mitigation).
Standard ERM (Average-Case):
All samples weighted equally ➔ Model learns majority shortcuts ➔ High average accuracy, poor tail performance
Tail-Risk Optimization:
High-loss samples prioritized ➔ Model forced to learn robust features ➔ Equitable performance across all groups
Compounding this issue, standard techniques designed to boost average performance can actively worsen outcomes for minority groups. In The Effects of Regularization and Data Augmentation are Class Dependent, Balestriero et al. reveal that uniform regularization strategies like data augmentation and weight decay often disproportionately degrade accuracy for specific minority classes. The global metric improves, hiding a "silent sacrifice" of underrepresented categories.
Shifting from Mean to Tail Risk
To build models that are reliable in worst-group scenarios, researchers are shifting the optimization objective away from the mean loss and toward the tail of the loss distribution. By changing how individual losses are aggregated, these frameworks force gradient descent to prioritize underperforming samples.
Distributionally Robust Optimization (DRO)
Instead of optimizing for the training distribution, DRO minimizes the worst-case expected loss across a set of possible distributions (an uncertainty set). Sagawa et al. demonstrate in Distributionally Robust Neural Networks that while DRO directly targets subpopulation disparities, overparameterized neural networks can still fail unless paired with strong regularization to bridge the generalization gap for minority groups. Minimax formulations have expanded into combinatorial routing problems (Yuan Jiang et al. in Learning to Solve Routing Problems via Distributionally Robust Optimization) and meta-learning (Collins et al. in Task-Robust Model-Agnostic Meta-Learning).
Tilted and Spectral Risks
Rather than taking a strict minimax approach, which can be overly sensitive to outliers, practitioners can "tilt" the optimization landscape. Szabó et al. introduce Tilted Cross-Entropy (TCE), a loss function that exponentially weights higher losses, allowing developers to dynamically tune how much the model prioritizes underperforming classes.
For broader guarantees, Ge et al. introduce SOREL, a stochastic gradient-based algorithm designed to minimize spectral risks—a family of risk-averse objectives that generalizes metrics like Conditional Value at Risk (CVaR) by targeting the tail of the loss distribution without relying on mathematical smoothing approximations.
Algorithmic Strategies for Subpopulation Fairness
When modifying the loss function directly is mathematically intractable or computationally expensive, researchers employ algorithmic interventions to balance the optimization process.
- Adaptive Reweighting: Instead of static weights, algorithms can dynamically adjust sample importance during training. In Fairness with Adaptive Weights, Chai proposes an adaptive reweighting framework that tracks and upweights hard-to-predict examples within each demographic group, preventing representation bias from eroding minority performance.
- Unsupervised Subclass Discovery: A core challenge in mitigating average-case bias is that minority subgroups are often unlabeled or hidden within broader, coarse-grained classes. To address this, Sohoni et al. introduce GEORGE in No Subclass Left Behind. This method uses unsupervised clustering on initial model features to automatically surface hidden subclasses, then retrains the model to maximize worst-case accuracy across all discovered groups.
- Contrastive and Multi-Task Learning: Rather than relying solely on classification losses, models can be forced to learn robust representations through auxiliary tasks. Zhang et al. develop Correct-N-Contrast (CNC) in Correct-N-Contrast: a Contrastive Approach for Improving Robustness to Spurious Correlations, which uses an intentionally biased model to identify hard, bias-conflicting samples and applies contrastive learning to force the network to map identical underlying concepts together, regardless of their background or environment.
By moving away from global averages and designing objectives that explicitly protect the tail of the distribution, these approaches ensure that machine learning models perform reliably for all subpopulations, rather than just the majority.