Dynamic Feature Alignment for Semi-supervised Domain Adaptation

Yu Zhang, Gongbo Liang, Nathan Jacobs

Abstract

Semi-supervised domain adaptation (SSDA) addresses the challenge of adapting models to a target domain using limited labeled data. Existing methods often fail to bridge the "intra-domain discrepancy" between labeled and unlabeled target samples. We present Dynamic Feature Alignment (DFA), a framework that aligns unlabeled target data to dynamically updated class prototypes stored in a memory bank. This approach provides stable, class-balanced targets for alignment, outperforming previous state-of-the-art methods on DomainNet and Office-Home benchmarks.

Introduction

Domain shift remains a significant hurdle in deploying machine learning models across varying environments. While Unsupervised Domain Adaptation (UDA) handles scenarios with no target labels, Semi-supervised Domain Adaptation (SSDA) leverages a small number of labeled target samples (e.g., 1 or 3 per class). Paradoxically, standard UDA techniques often underperform in SSDA settings due to the "intra-domain discrepancy"—the gap between sparse labeled target data and the broader unlabeled target distribution. Furthermore, traditional batch-based alignment is susceptible to class imbalance, leading to suboptimal feature mapping.

Dynamic Feature Alignment (DFA)

To overcome these challenges, we propose Dynamic Feature Alignment (DFA). Instead of direct feature-to-feature alignment within mini-batches, DFA utilizes a Dynamic Memory Bank to maintain representative "class prototypes."

Overview of the Dynamic Feature Alignment (DFA) framework, illustrating the interaction between labeled source, labeled target, and unlabeled target data through a memory bank.

Key Components

  • Class Prototypes: The memory bank stores a stable feature vector for each class, acting as a global reference point for alignment.
  • Two-Tier Memory Update: To ensure prototype stability, an intermediate bank stores the most recent correctly classified feature for each class. These are then integrated into the main bank via an Exponentially Weighted Moving Average (EWMA).
  • MMD Alignment: The model minimizes the Maximum Mean Discrepancy (MMD) between unlabeled target features and these stable prototypes.
  • Selective Pseudo-labeling: Labels are assigned to unlabeled data only when the model exhibits high confidence (low entropy) and the feature is proximal to the corresponding prototype.

Experimental Evaluation

We evaluated DFA on the DomainNet and Office-Home benchmarks using AlexNet and ResNet-34 backbones in 1-shot and 3-shot settings.

Performance comparison on the DomainNet dataset (3-shot setting) across various adaptation scenarios, showing DFA outperforming previous state-of-the-art methods.

Key Results

  • DomainNet: In the 3-shot ResNet setting, DFA achieved a mean accuracy of 72.8%, a 1.1% improvement over the previous state-of-the-art (APE). In the 1-shot setting, it reached 68.9%, outperforming baselines by 1.3%.
  • Office-Home: DFA maintained the highest overall mean accuracy of 74.4%, leading in 8 out of 12 scenarios.
  • Qualitative Analysis: t-SNE visualizations confirmed that DFA produces tighter, more distinct clusters for target features as training progresses, effectively bridging the domain gap.

Conclusion

Dynamic Feature Alignment offers a robust solution for SSDA by replacing noisy batch-level alignment with stable, prototype-based alignment. By focusing on reducing intra-domain discrepancy and maintaining class-balanced prototypes, DFA sets a new standard for performance on large-scale domain adaptation tasks. Future work may explore automated hyperparameter tuning and extending this framework to complex tasks like object detection.

Summary updated

BibTeX
@inproceedings{zhang2021dynamic,
  author = {Yu Zhang and Gongbo Liang and Nathan Jacobs},
  title = {{Dynamic Feature Alignment for Semi-supervised Domain Adaptation}},
  booktitle = {BMVC},
  year = {2021},
  url = {https://openreview.net/forum?id=tJpjMo-sdE},
}
Figure