Abstract
Machine learning models often perform poorly under subpopulation shifts in the data distribution. Developing methods that allow machine learning models to better generalize to such shifts is crucial for safe deployment in real-world settings. In this paper, we develop a family of group-aware prior (GAP) distributions over neural network parameters that explicitly favor models that generalize well under subpopulation shifts. We design a simple group-aware prior that only requires access to a small set of data with group information and demonstrate that training with this prior yields state-of-the-art performance—even when only retraining the final layer of a previously trained non-robust model. Group aware-priors are conceptually simple, complementary to existing approaches, such as attribute pseudo labeling and data reweighting, and open up promising new avenues for harnessing Bayesian inference to enable robustness to subpopulation shifts.
Introduction: The Challenge of Subpopulation Shifts
Standard machine learning models are typically trained using Empirical Risk Minimization (ERM), which focuses on minimizing average error across a dataset. While effective for the majority of cases, ERM often fails on minority subgroups due to subpopulation shifts. This occurs when a model learns "spurious correlations"—for instance, identifying a bird as a "waterbird" simply because there is water in the background. Such failures are critical in sensitive applications like healthcare or hiring, where high accuracy on every demographic group is a necessity.
Methods: The Group-Aware Prior (GAP) Framework
Researchers from NYU and Meta FAIR have introduced Group-Aware Priors (GAP), a Bayesian framework designed to mitigate these failures. Instead of standard optimization tricks, GAP uses a data-driven prior to steer the model toward parameters that perform well on minority groups.
The GAP methodology follows a three-step process:
- ERM Pre-training: A model is first trained on the full dataset without group information to establish a baseline.
- Prior Construction: Using a small, group-labeled validation set, the researchers construct a prior that heavily upweights minority groups.
- Refitting: The model is retrained using an objective that combines standard likelihood with the GAP. This process favors "flat" regions of the loss landscape, ensuring that slight perturbations in model weights do not degrade performance on minority groups.
One of the most powerful aspects of GAP is its flexibility: it can be used to fine-tune an entire network or, more efficiently, just the final linear layer of a pre-trained ERM model.
Datasets and Evaluation
The team evaluated GAP on three classic robustness benchmarks where spurious correlations are prevalent:
- Waterbirds: Bird types (water vs. land) correlated with backgrounds.
- CelebA: Predicting hair color (blond vs. non-blond) where gender acts as a distracting spurious feature.
- MultiNLI: A natural language processing task where negation words are spuriously linked to specific logical categories.
Key Results and Insights
GAP set new state-of-the-art (SOTA) records for Worst-Group Accuracy (WGA), which measures performance on the most challenging subgroup:
- Waterbirds: Achieved 93.8% WGA, outperforming popular baselines like Group-DRO (91.4%).
- CelebA: Reached 90.2% WGA.
- MultiNLI: Achieved 77.8% WGA.
A key insight from the study is the power of the last layer. Retraining only the final layer with a GAP prior often outperformed complex methods that retrain entire networks. This suggests that standard models often learn the necessary features for minority groups but fail to weight them correctly during the final classification stage.
Conclusion
By framing robustness as a Bayesian inference problem, GAP provides a principled and modular way to ensure fairness across subpopulations. While the method currently requires a small amount of group-labeled data, it demonstrates that "flatness" in the loss landscape is a strong indicator of group robustness, opening new avenues for reliable AI deployment.
Summary updated