Gradient Regularized Contrastive Learning for Continual Domain Adaptation

Shixiang Tang, Peng Su, Dapeng Chen, Wanli Ouyang

TL;DR

This paper introduces Gradient Regularized Contrastive Learning (GRCL), a novel method for continual domain adaptation. GRCL tackles the challenge of adapting models to new, unseen data domains sequentially without forgetting previously learned information, achieving state-of-the-art performance.

What Problem Does This Paper Address?

In real-world applications, machine learning models often encounter data from new domains that differ from their training data. This "domain shift" can significantly degrade performance. Continual Domain Adaptation (Continual DA) addresses this by assuming a model is trained on a labeled source domain and then sequentially presented with unlabeled target domains. The key challenges here are:

  1. Domain Shift: The distribution of data changes between domains, making it hard for the model to generalize.
  2. Catastrophic Forgetting: When adapting to a new domain, the model tends to forget what it learned from previous domains.

Traditional domain adaptation methods often require retraining from scratch with all available data, which is inefficient. Continual DA aims to adapt incrementally, preserving knowledge from past domains.

Methods

The proposed method, Gradient Regularized Contrastive Learning (GRCL), builds upon contrastive learning and introduces two novel gradient regularization constraints:

  • Contrastive Learning Framework: GRCL uses a contrastive loss to learn domain-invariant representations. This involves pulling similar samples closer in an embedding space and pushing dissimilar ones apart. A feature bank is maintained to store representative features from the source and previously seen target domains.
  • Source Discriminative Constraint: This constraint ensures that the model's update direction does not increase the classification loss on the source domain. Since source domain labels are reliable, this preserves the discriminative ability of source features, which surprisingly aids adaptation to new target domains.
  • Target Memorization Constraint: This constraint prevents the model's update from increasing the classification loss on old target domains. This is crucial for mitigating catastrophic forgetting and retaining performance on previously learned domains.

Instead of directly optimizing a complex objective, GRCL formulates these constraints as a quadratic programming problem, which is solved efficiently in the dual space.

Datasets

The paper evaluates GRCL on three standard domain adaptation benchmarks:

  • Digits: A collection of five digit datasets (MNIST, MNIST-M, USPS, SynNum, SVHN). A continual DA task is constructed by sequentially adapting from SynNum to MNIST, then MNIST-M, USPS, and finally SVHN.
  • DomainNet: A large-scale dataset with 345 categories. A continual DA task is created using five domains: Clipart, Real, Infograph, Sketch, and Painting.
  • Office-Caltech: A dataset combining images from Office-31 and Caltech-256. A continual DA task is set up with DSLR, Amazon, WebCam, and Caltech as sequential target domains.

Key Insights and Ideas

  • Source Domain Knowledge is Key: Preserving the discriminative power of the source domain, even when adapting to new target domains, significantly boosts adaptation performance. This is counter-intuitive, as one might expect to focus solely on target domain adaptation.
  • Gradient Regularization for Forgetting: Directly constraining the gradient of the model's update to not increase losses on old domains is a more effective way to combat catastrophic forgetting than traditional replay or multitask learning approaches.
  • Adaptive Trade-offs: GRCL's approach of solving a quadratic program to determine the update direction allows for adaptive weighting of different loss components (contrastive loss, source constraint, target constraint) in each iteration, rather than relying on manually tuned hyperparameters.
  • Contrastive Learning Benefits from Memory: The performance of the contrastive learning component of GRCL improves with larger memory banks and longer training schedules, aligning with prior findings in contrastive learning research.

Key Results

GRCL consistently outperforms existing state-of-the-art methods across all three benchmarks in terms of average accuracy (ACC). Notably, it achieves significantly better backward transfer (BWT), indicating a strong ability to overcome catastrophic forgetting.

  • Digits: GRCL achieves 85.34% ACC and -0.67% BWT.
  • DomainNet: GRCL achieves 37.74% ACC and -0.67% BWT.
  • Office-Caltech: GRCL achieves 87.23% ACC and 0.05% BWT.

Memory-based methods (including GRCL) generally perform better than memory-free methods, showing improvements of 2-5% in ACC and 3-5% in BWT. GRCL specifically demonstrates minimal forgetting and even positive backward transfer on the Office-Caltech benchmark.

Limitations and Future Work

  • Memory Size: While GRCL benefits from larger memory banks, the paper notes that performance saturates beyond a certain size (e.g., 1024 for Digits).
  • Computational Cost: The paper mentions that directly solving the quadratic programming problem for a large neural network can be computationally intensive, although their dual space solution is efficient.
  • Future Work: The authors suggest that their gradient regularization approach could be extended to other continual learning scenarios beyond domain adaptation. Further exploration of adaptive trade-off mechanisms and memory management strategies could also be beneficial.

Summary updated

Abstract

Human beings can quickly adapt to environmental changes by leveraging learning experience. However, adapting deep neural networks to dynamic environments by machine learning algorithms remains a challenge. To better understand this issue, we study the problem of continual domain adaptation, where the model is presented with a labelled source domain and a sequence of unlabelled target domains. The obstacles in this problem are both domain shift and catastrophic forgetting. We propose Gradient Regularized Contrastive Learning (GRCL) to solve the obstacles. At the core of our method, gradient regularization plays two key roles: (1) enforcing the gradient not to harm the discriminative ability of source features which can, in turn, benefit the adaptation ability of the model to target domains; (2) constraining the gradient not to increase the classification loss on old target domains, which enables the model to preserve the performance on old target domains when adapting to an in-coming target domain. Experiments on Digits, DomainNet and Office-Caltech benchmarks demonstrate the strong performance of our approach when compared to the other state-of-the-art methods.
BibTeX
@inproceedings{tang2021gradient,
  author = {Shixiang Tang and Peng Su and Dapeng Chen and Wanli Ouyang},
  title = {{Gradient Regularized Contrastive Learning for Continual Domain Adaptation}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2021},
  doi = {10.1609/aaai.v35i3.16370},
}
Figure