Label-Efficient Group Robustness via Out-of-Distribution Concept Curation

Yiwei Yang, Anthony Z. Liu, Robert Wolfe, Aylin Caliskan, Bill HoweCVPR 2024 · OpenReview

Abstract

Deep neural networks are prone to capture correlations between spurious attributes and class labels, leading to low accuracy on some combinations of class labels and spurious attribute values. When a spurious attribute represents a protected class, these low-accuracy groups can manifest discriminatory bias. Existing methods attempting to improve worst-group accuracy assume the training data, validation data, or both are reliably labeled by the spurious attribute. But a model may be perceived to be biased towards a concept that is not represented by pre-existing labels on the training data. In these situations, the spurious attribute must be defined with external information. We propose Concept Correction, a framework that represents a concept as a curated set of images from any source, then labels each training sample by its similarity to the concept set to control spurious correlations. For example, concept sets representing gender can be used to measure and control gender bias even without explicit labels. We demonstrate and evaluate an instance of the framework as Concept DRO, which uses concept sets to estimate group labels, then uses these labels to train with a state of the art distributively robust optimization objective. We show that Concept DRO outperforms existing methods that do not require labels of spurious attributes by up to 33.1 % on three image classification datasets and is competitive with the best methods that assume access to labels. We consider how the size and quality of the concept set influences performance and find that even smaller, manually curated sets of noisy AI-generated images are effective at controlling spurious correlations, suggesting that high-quality, reusable concept sets are easy to create and effective in reducing bias.

Introduction

Deep neural networks often rely on "shortcuts" or spurious correlations to achieve high average accuracy. For instance, if a dataset of doctors is predominantly male, a model might learn to associate "maleness" with the label "doctor" rather than identifying medical expertise. This leads to poor performance on minority groups, such as female doctors. Standard solutions like Group Distributionally Robust Optimization (Group DRO) mitigate this by optimizing for the worst-performing group. However, Group DRO requires every training image to be manually labeled with its spurious attribute (e.g., gender), which is often prohibitively expensive and slow. Existing label-free methods often struggle with complex real-world data or still require labels for validation sets.

Methodology: Concept DRO

The authors introduce Concept Correction, specifically implemented as Concept DRO (CDRO). This framework leverages the insight that even biased models, trained via standard Empirical Risk Minimization (ERM), learn features of spurious attributes in their internal layers. The CDRO workflow follows a four-step process to mitigate bias without large-scale manual labeling:

  1. Train an ERM Model: A standard model is trained on the biased dataset.
  2. Learn a Linear CAV: A human provides a small "concept set" (e.g., 50 images of land) and a "contrastive set" (e.g., 50 images of water). These images can be sourced externally or generated via AI. A Concept Activation Vector (CAV) is then learned in the model's feature space.
  3. Infer Pseudo-Group Labels: The unlabeled training set is projected onto this CAV. A Gaussian Mixture Model (GMM) clusters these scores to automatically assign pseudo-labels representing the spurious attribute.
  4. Run Group DRO: A new model is trained using the Group DRO objective, utilizing these inferred pseudo-labels instead of manual ones.
The four-step workflow of Concept DRO (CDRO) for mitigating model bias.

Experimental Results

The researchers evaluated CDRO on three benchmarks: CMNIST (color bias), Waterbirds (background bias), and CelebA (gender/hair color bias). They demonstrated that out-of-distribution (OOD) images, including those generated by Stable Diffusion, can effectively serve as concept sets.

CDRO achieved significant gains in worst-group accuracy. On the CelebA dataset, it improved performance by 33.1% compared to the unsupervised baseline GEORGE. As shown in the table below, CDRO consistently outperforms other methods that do not require group labels and remains competitive with supervised methods that have access to full manual labels.

Performance comparison of CDRO against various baselines across three datasets. CDRO (ours) shows strong performance without requiring training or validation group labels.

Key Insights and Discussion

  • Label Efficiency: CDRO is remarkably efficient, achieving high performance with as few as 4 to 50 curated concept images, whereas traditional methods might require tens of thousands of validation labels.
  • Robustness to Concept Source: The framework is resilient to the source and quality of concept images. Synthetic images of "land" or "gendered features" worked nearly as well as real images from the target distribution.
  • Internal Knowledge Probing: The success of CDRO confirms that biased models "accidentally" learn the features of spurious attributes. CDRO provides a mechanism to extract and utilize this latent knowledge.

Conclusion and Limitations

Concept DRO offers a scalable, label-efficient path to improving model robustness. However, it requires a human to identify the potential bias beforehand to curate the appropriate concept sets. Additionally, it depends on the initial ERM model's ability to capture the spurious feature in its penultimate layer. Future research may focus on automating bias discovery or utilizing text-based probes to further simplify the concept curation process.

Summary updated

Abstract

Deep neural networks are prone to capture correlations between spurious attributes and class labels, leading to low accuracy on some combinations of class labels and spurious attribute values. When a spurious attribute represents a protected class, these low-accuracy groups can manifest discriminatory bias. Existing methods attempting to improve worst-group accuracy assume the training data, validation data, or both are reliably labeled by the spurious attribute. But a model may be perceived to be biased towards a concept that is not represented by pre-existing labels on the training data. In these situations, the spurious attribute must be defined with external information. We propose Concept Correction, a framework that represents a concept as a curated set of images from any source, then labels each training sample by its similarity to the concept set to control spurious correlations. For example, concept sets representing gender can be used to measure and control gender bias even without explicit labels. We demonstrate and evaluate an instance of the framework as Concept DRO, which uses concept sets to estimate group labels, then uses these labels to train with a state of the art distributively robust optimization objective. We show that Concept DRO outperforms existing methods that do not require labels of spurious attributes by up to 33.1 % on three image classification datasets and is competitive with the best methods that assume access to labels. We consider how the size and quality of the concept set influences performance and find that even smaller, manually curated sets of noisy AI-generated images are effective at controlling spurious correlations, suggesting that high-quality, reusable concept sets are easy to create and effective in reducing bias.
BibTeX
@inproceedings{yang2024label,
  author = {Yiwei Yang and Anthony Z. Liu and Robert Wolfe and Aylin Caliskan and Bill Howe},
  title = {{Label-Efficient Group Robustness via Out-of-Distribution Concept Curation}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2024},
  url = {https://openreview.net/forum?id=OcivinauZa},
}
Figure