Continual Learning in the Presence of Spurious Correlation

Donggyu Lee, Sangwon Jung, Taesup MoonarXiv 2023 · DOI · arXiv

Abstract

Continual Learning (CL) aims to enable models to learn from a sequence of tasks without catastrophic forgetting. However, real-world datasets often contain spurious correlations—biases where models associate labels with irrelevant features. This paper investigates how these biases behave in sequential learning, revealing that they "leak" both forward and backward through the task sequence. To mitigate this, the authors propose Group-class Balanced Greedy Sampling (BGS), a plug-in method that retrains the classification layer using a balanced memory of past examples.

The Challenge: Bias Transfer in Sequential Learning

Most CL research focuses on the stability-plasticity dilemma: balancing the retention of old knowledge with the acquisition of new information. This paper identifies a new dimension to this challenge: the transfer of dataset bias. When a model learns a biased task, that bias can infect its performance on both future tasks (forward transfer) and previously learned tasks (backward transfer).

Figure 1: Illustration of forward and backward transfer of bias. A biased task $T_t$ causes the model to make incorrect, bias-driven predictions on both previous ($T_{t-1}$) and subsequent ($T_{t+1}$) tasks.

For example, if a model learns to identify "young" faces in a dataset where most young people are female, it may develop a gender bias. This bias doesn't just affect the current task; it can lead the model to misclassify young males in past or future tasks as "old" if they don't fit the learned spurious correlation.

Methodology: Group-class Balanced Greedy Sampling (BGS)

To address bias leakage, the authors introduce Group-class Balanced Greedy Sampling (BGS). This approach is designed to be a "plug-and-play" addition to existing CL algorithms. It operates on two main principles:

  1. Balanced Exemplar Memory: Unlike standard rehearsal methods that balance memory by class (e.g., "dog" vs. "cat"), BGS balances memory by both class and group (e.g., "dog in grass" vs. "dog on sofa"). This ensures the model sees a representative variety of features for each label.
  2. Last-Layer Retraining: After a task is learned using a standard CL method, BGS freezes the feature extractor and retrains only the final classification head using the balanced memory. This strategy targets the "head" of the network, teaching it to ignore biased features extracted by the "body."

Key Insights: The Stability-Plasticity-Bias Trilemma

The authors' investigation into bias transfer yielded several critical insights:

  • Contagious Bias: Bias is not isolated. Forward transfer occurs when a biased past task influences future learning, while backward transfer occurs when a new biased task retroactively corrupts old logic.
  • Trade-off Effects: The choice of CL method significantly impacts bias transfer. Stability-oriented methods (like EWC) are more prone to forward bias transfer as they preserve early biased features. Conversely, plasticity-oriented methods (like Fine-tuning) are more susceptible to backward bias transfer, quickly overwriting old knowledge with new, biased correlations.
  • Accumulation: In longer task sequences, shared biases across tasks compound, making the model increasingly reliant on spurious features.

Experimental Results

The BGS method was evaluated on three benchmarks: Split CIFAR-100S (color bias), CelebA (gender bias), and Split ImageNet-100 (watermark bias).

Figure 2: Performance comparison across different CL scenarios. The plots show the relationship between stability/plasticity (y-axis) and bias metrics like BMR or DCA (x-axis) for various algorithms and bias levels.

As shown in Figure 2, the degree of bias (BMR or DCA) varies significantly across different CL categories. The proposed BGS method consistently reduces these bias metrics:

  • Split CIFAR-100S: Adding BGS to the LwF algorithm reduced the Bias-flipped Mis-classification Rate (BMR) from 28.00% to 16.82% while improving average accuracy.
  • CelebA: For Experience Replay (ER), BGS reduced gender bias from 37.68% to 30.49% with minimal impact on accuracy.

Conclusion and Future Directions

This research highlights a significant but overlooked risk in continual learning: the propagation of spurious correlations. While BGS provides an effective and simple mitigation strategy by retraining the classification head, it currently requires group labels which may not always be available. Future work should focus on debiasing the feature extractor itself and exploring scenarios where group labels are unknown.

Summary updated

Abstract

Most continual learning (CL) algorithms have focused on tackling the stability-plasticity dilemma, that is, the challenge of preventing the forgetting of previous tasks while learning new ones. However, they have overlooked the impact of the knowledge transfer when the dataset in a certain task is biased - namely, when some unintended spurious correlations of the tasks are learned from the biased dataset. In that case, how would they affect learning future tasks or the knowledge already learned from the past tasks? In this work, we carefully design systematic experiments using one synthetic and two real-world datasets to answer the question from our empirical findings. Specifically, we first show through two-task CL experiments that standard CL methods, which are unaware of dataset bias, can transfer biases from one task to another, both forward and backward, and this transfer is exacerbated depending on whether the CL methods focus on the stability or the plasticity. We then present that the bias transfer also exists and even accumulate in longer sequences of tasks. Finally, we propose a simple, yet strong plug-in method for debiasing-aware continual learning, dubbed as Group-class Balanced Greedy Sampling (BGS). As a result, we show that our BGS can always reduce the bias of a CL model, with a slight loss of CL performance at most.
BibTeX
@misc{lee2023continual,
  author = {Donggyu Lee and Sangwon Jung and Taesup Moon},
  title = {{Continual Learning in the Presence of Spurious Correlation}},
  year = {2023},
  eprint = {2303.11863},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2303.11863},
}
Figure