Concept Correlation and Its Effects on Concept-Based Models

Lena Heidemann, Maureen Monnet, Karsten RoscherWACV 2023

TL;DR

This paper investigates how correlations between concepts in image datasets can mislead concept-based models, causing them to rely on proxies rather than true visual features. The authors propose new evaluation methods beyond simple accuracy to detect these issues and demonstrate that high correlations can significantly degrade model performance, especially with limited training data. They also introduce promising mitigation techniques to improve concept learning in the presence of such correlations.

What Problem Does This Paper Address?

Concept-based models, like Concept Bottleneck Models (CBMs), aim to make deep learning models more interpretable by learning explicit, high-level concepts that are then used to predict the final class. While these models often achieve competitive accuracy, a critical question remains: are the learned concepts truly representative of the visual features they're supposed to capture, or are they learning shortcuts? This paper tackles the problem of "spurious correlations" between concepts within a dataset. If two concepts are highly correlated (e.g., "has a red beak" and "is a robin"), a model might learn to predict "is a robin" simply by detecting "has a red beak," without actually understanding what a robin looks like. This is particularly problematic for safety-critical applications where reliable concept understanding is crucial. The paper aims to quantify this problem and explore solutions.

Methods

The authors propose several methods to evaluate concept-based models beyond standard test accuracy:

  • Concept Removal: This involves removing a concept from an image and observing if the model's prediction for that concept changes. This is done quantitatively on synthetically generated data and qualitatively on real-world images.
  • Pointing Game: This technique uses saliency maps (generated via Guided Grad-CAM) and ground truth concept masks to assess if the model's attention is focused on the correct visual regions for a concept.
  • Difference in Test Accuracy: This method compares a model's accuracy on subsets of data where concepts are either both present/absent (dependent samples) versus only one is present (independent samples). A significant difference suggests the model is not learning the concept independently.

For mitigation, they explore:

  • Training One Model for Each Concept: Instead of a single model predicting all concepts, separate binary classifiers are trained for each concept. This aims to isolate concepts and reduce side effects from shared feature extractors.
  • Weighting the Loss of Independent Samples: To encourage models to learn both concepts in a correlated pair, the loss for samples where only one concept is present is given higher weight during training.

Datasets

The paper uses the following datasets:

  • Caltech-UCSD Birds-200-2011 (CUB): A real-world dataset with 11,788 images and 200 bird species, featuring attribute annotations for concepts.
  • German Traffic Sign Recognition Benchmark (GTSRB): A dataset of 50,000 images with 43 traffic sign classes. The authors define 43 concepts for this dataset, including colors, shapes, and symbols.
  • Synthetically Generated 3D Shapes: Datasets generated using the CLEVR dataset generator. These datasets allow for precise control over concept correlations and image generation, enabling a deeper analysis of concept correlation effects. Concepts include shapes (cube, sphere, cylinder, cone, torus, icosphere) with variations in size, material, and color.

Key Insights and Ideas

  • Test accuracy is insufficient: Simply reporting the accuracy of a concept prediction is not enough to guarantee that the model has learned the concept visually.
  • High correlation is detrimental: As concept correlation approaches 1 (or -1), the model's ability to learn concepts independently degrades significantly, especially when training data is limited.
  • Models learn proxies: Highly correlated concepts can lead models to use one concept as a proxy for another, failing to learn the true visual representation.
  • Side effects on unrelated concepts: Concept correlations can negatively impact the learning of other, seemingly unrelated concepts, likely due to shared visual features.
  • Mitigation is possible: Training individual concept models and strategically weighting losses can help alleviate the negative impacts of concept correlation.

Key Results

  • On CUB and GTSRB, the difference in balanced test accuracy between dependent and independent samples was largest for concepts with correlations close to 1 and -1, indicating learning issues.
  • On synthetic 3D shapes datasets, increasing concept correlation led to a drop in accuracy for one of the concepts, particularly when using less training data. For instance, with a correlation of 0.89 and 50% training data, the accuracy for "icosphere" dropped significantly.
  • The proposed mitigation technique of weighting the loss for independent samples improved accuracy for correlated concepts. For example, for a torus-icosphere pair with 0.88 correlation, accuracy increased from 93.4% to 97.0% with loss weighting.
  • Training separate models for each concept helped reduce the negative side effects on unrelated concepts compared to a shared concept model.

Limitations and Future Work

  • Limitations of evaluation metrics: The Pointing Game requires ground truth concept masks, which are not available for real-world datasets like CUB and GTSRB. Saliency map limitations are also acknowledged.
  • Real-world complexity: The proposed mitigation techniques were tested on 3D shapes datasets with a single highly correlated concept pair. Real-world datasets often have multiple highly correlated pairs, requiring more complex strategies.
  • Future Work:
    • Analyze more complex, non-pairwise concept relations.
  • Develop more sophisticated weighting or upsampling methods for multiple correlated concepts.
  • Explore generating synthetic data with independent concepts.
  • Investigate guiding models with localization information for a few samples.
  • Address the computational cost of training individual concept models for real-time applications.

Summary updated

BibTeX
@inproceedings{heidemann2023concept,
  author = {Lena Heidemann and Maureen Monnet and Karsten Roscher},
  title = {{Concept Correlation and Its Effects on Concept-Based Models}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2023},
}
Figure