Abstract
Existing methods for isolating hard subpopulations and spurious correlations in datasets often require human intervention, making them labor-intensive and dataset-specific. We present a scalable method for automatically distilling a model's failure modes by harnessing linear classifiers to identify consistent error patterns. This induces a natural representation of failure modes as directions within a shared vision-language feature space. We demonstrate that this framework allows for the discovery and automatic captioning of challenging subpopulations. Furthermore, by combining our framework with off-the-shelf diffusion models, we generate synthetic "hard" images to remedy these failure modes through targeted data augmentation.
Introduction
Deep learning models frequently rely on "spurious correlations"—shortcuts in training data that fail to generalize to real-world scenarios. For instance, a model might correctly identify "dogs" only when they appear on grass, failing when the background changes. Identifying these "failure slices" or hard subpopulations typically requires exhaustive manual data exploration. While automated methods exist, they often lack interpretability, providing a list of errors without explaining the underlying cause. We address this by proposing a framework that extracts human-understandable patterns of model errors automatically.
Methodology: Failure as a Vector
Our approach performs a "meta-analysis" of a trained base model's mistakes by leveraging the structured latent space of CLIP.
The pipeline consists of four primary stages:
- Embedding: Images from a validation set are encoded into a d-dimensional space using a frozen CLIP encoder. CLIP is ideal because its latent space aligns visual features with natural language.
- Error Prediction: For each class, we train a linear Support Vector Machine (SVM) to distinguish between images the base model classifies correctly and those it misses.
- Defining Failure Directions: The normal vector to the SVM’s decision boundary defines a "failure direction." Moving along this vector transitions from "easy" examples to "hard" ones.
- Interpretation and Intervention: We interpret these directions by scoring candidate text captions against the failure vector. To fix the model, we use the failure direction to guide a diffusion model, generating synthetic "hard" examples for targeted data augmentation.
Using a linear SVM provides a strong inductive bias, forcing the system to find the simplest, most consistent failure patterns—typically those most interpretable to humans, such as background or color.
Experimental Evaluation
We validated our framework across diverse datasets, including CelebA, CIFAR-10/100, ImageNet, and ChestX-ray14.
Subpopulation Discovery and Captioning
On the CelebA dataset, we investigated a planted spurious correlation where age is correlated with gender. Our method significantly outperformed baselines in flagging hard subpopulations, achieving high precision in identifying challenging groups like young women in an "Old" class.
As shown in the figure above, the SVM identifies "hard" exemplars and provides descriptive captions. We also found a direct correlation between the SVM's cross-validation accuracy and the actual strength of the spurious correlation in the training data, providing a quantitative metric for model bias.
Real-World Failure Modes
In CIFAR-10, the method automatically discovered that models struggled with "white cats on grass" while finding "brown cats" easy. In medical imaging (ChestX-ray14), it identified that models were relying on patient positioning (Anterior-Posterior vs. Posterior-Anterior) as a confounder for diagnosis.
Targeted Intervention
Fine-tuning models on synthetic "hard" images generated via the failure direction proved more effective than generic data augmentation. By generating targeted images, we observed measurable improvements in accuracy on the identified challenging subpopulations.
Discussion and Future Work
While effective, our current method focuses on the single most prominent failure mode per class. Real-world classes often exhibit multiple, distinct reasons for failure. Future work will explore disentangling multiple failure directions. Additionally, moving from predefined candidate captions to free-form generation using Large Language Models (LLMs) could provide richer interpretations. Finally, the efficacy of our intervention remains tied to the generative capabilities of the underlying diffusion models.
Code available at: https://github.com/MadryLab/failure-directions
Summary updated