TL;DR
This paper investigates why standard deep learning models often fail on under-represented subgroups, even when their internal representations contain useful information. They discover that different robust training methods learn remarkably similar representations to standard training. The key to improving subgroup performance lies not in better representations, but in how we adapt them post-hoc, with the validation set's distribution being critically important.
What Problem Does This Paper Address?
Deep learning models trained with Expected Risk Minimization (ERM) often exhibit significant performance disparities across different subgroups within the data. For instance, a model trained to identify animals might perform well on common breeds but poorly on rarer ones. While specialized algorithms exist to improve performance on these under-represented groups (often by leveraging subgroup information during training), a previous study by Menon et al. suggested that even standard ERM models might learn representations that already contain the necessary information to perform well across all subgroups. This raises a crucial question: if the information is already there, why the performance gap, and how can we effectively extract it? This paper aims to disentangle the roles of representation learning and prediction adaptation in achieving subgroup robustness.
Methods
The authors propose a three-stage "Train-Adapt-Test" procedure:
- Train: A model is trained on a dataset with an unbalanced subgroup distribution using standard ERM. This yields both a prediction function and an underlying representation function.
- Adapt: Post-hoc adaptation is performed only on the learned representations. This involves training a new, simpler classifier (e.g., logistic regression, k-NN, or vector quantization) on these representations using a separate validation set. Crucially, this adaptation step does not use subgroup information.
- Test: The adapted model (representation function + post-hoc classifier) is evaluated on held-out test data, measuring performance across all subgroups.
They compare ERM with two robust training methods: Group Distributionally Robust Optimization (GDRO) and Invariant Risk Minimization (IRM). To analyze representations, they use Singular Vector Canonical Correlation Analysis (SVCCA) to measure similarity between representations learned by different models.
Datasets
The primary dataset used is the semi-synthetic Waterbirds dataset. This dataset is constructed by combining bird images (CUB dataset) with backgrounds (Places-365 dataset), creating four subgroups based on bird type (land/water) and background type (land/water). The key characteristic of Waterbirds is that the training set has a strong correlation between bird and background (e.g., water birds are often on water backgrounds), leading to subgroup imbalances. The dataset includes a separate, more subgroup-balanced validation set, which is crucial for their post-hoc adaptation experiments. For initialization, they use models pre-trained on ImageNet.
Key Insights and Ideas
- Representation Similarity: Surprisingly, different training paradigms (ERM, GDRO, IRM) learn remarkably similar internal representations. The paper argues that the "feature co-discovery" is more prominent than "feature reuse" when comparing models trained with and without subgroup information.
- Post-hoc Adaptation is Key: The performance gap between ERM and robust methods isn't primarily due to ERM failing to learn useful features. Instead, the effectiveness of post-hoc adaptation on ERM's representations can bridge this gap.
- Validation Set Distribution Matters Most: The distribution of the post-hoc validation set is paramount. A validation set that is more balanced across subgroups (like the one provided with Waterbirds) allows for much better adaptation and improved worst-group accuracy compared to using a validation set with a distribution similar to the imbalanced training set.
- Clustering-based Classifiers Excel: Among the post-hoc adaptation methods explored (logistic regression, k-NN, Vector Quantization), clustering-based approaches like Vector Quantization (VQ) show particular promise, often outperforming simpler linear classifiers.
Key Results
- SVCCA similarity scores between representations learned by ERM, GDRO, and IRM on the Waterbirds dataset are consistently high (~0.9), indicating that these methods learn practically identical features. This similarity is comparable to models trained with different random seeds.
- By using the subgroup-balanced Waterbirds validation set for post-hoc adaptation, a standard ERM model can achieve worst-group accuracy comparable to specialized robust methods like GDRO/IRM, which were trained with subgroup information.
- When the post-hoc validation set's minority subgroup percentage is reduced from 100% (balanced) to 5% (mimicking training imbalance), worst-group accuracy drops significantly.
- Vector Quantization (VQ) as a post-hoc classifier consistently improved worst-group accuracy over the original ERM model and linear adaptation, even when using a validation set with only 5% minority examples.
Limitations and Future Work
- Dataset Specificity: While Waterbirds is a useful benchmark, the findings might not generalize directly to all real-world datasets, which can be more complex.
- Post-hoc Classifier Choice: The paper explores a few post-hoc classifiers. Further investigation into more advanced adaptation techniques could yield even greater improvements.
- Future Directions: The authors suggest that learning better representations for subgroup classification is still a promising direction. However, their work strongly emphasizes that harnessing the richness of existing ERM representations through effective post-hoc adaptation is a highly viable and potentially more efficient path to achieving subgroup robustness. They also hint at exploring more complex adaptation methods beyond simple classifiers.
Summary updated