Adaptive Adversarial Network for Source-free Domain Adaptation

Haifeng Xia, Handong Zhao, Zhengming Ding

Introduction

Traditional Unsupervised Domain Adaptation (UDA) relies on simultaneous access to a labeled source dataset and an unlabeled target dataset. However, real-world constraints such as data privacy (e.g., medical records) and storage limitations (e.g., massive datasets on edge devices) often necessitate Source-free Domain Adaptation (SFDA). In SFDA, a model must adapt to a new target domain using only a pre-trained source model, without access to the original source data.

The primary challenge in SFDA is the "frozen" nature of the source model. When target data deviates significantly from the source distribution—forming a "source-dissimilar" set—accuracy typically drops. Existing methods often struggle to re-align these dissimilar samples without the source data as a reference.

Methodology

The proposed Adaptive Adversarial Network ($A^2Net$) addresses these challenges through three integrated components:

Overview of the $A^2Net$ architecture, highlighting the dual-classifier adversarial mechanism, self-supervised rotation task, and Contrastive Category-wise Matching (CCM).
  1. Adaptive Adversarial Inference: $A^2Net$ employs a dual-classifier strategy. It utilizes a frozen source classifier ($C_s$) and a learnable target classifier ($C_t$). Target samples are adaptively categorized into "source-similar" and "source-dissimilar" sets. An adversarial game is then played: the feature generator attempts to align "hard" (dissimilar) samples with "easy" (similar) ones, while the classifiers attempt to distinguish between them. This discrepancy-based adversarial signal effectively aligns the target distribution without needing source domain labels.
  2. Contrastive Category-wise Matching (CCM): To enhance intra-class compactness, $A^2Net$ performs contrastive learning on target image pairs. It uses a dynamic "easy-to-hard" curriculum, where similarity thresholds are adjusted over time to include increasingly ambiguous pairs as training progresses.
  3. Self-Supervised Rotation: A rotation prediction task (0°, 90°, 180°, 270°) is incorporated as a pretext task. This forces the feature extractor to capture robust geometric and structural information from the target images, improving feature representation.

Key Results

$A^2Net$ was evaluated on standard benchmarks including Office-31, Office-Home, and VisDA. It consistently achieved state-of-the-art performance among source-free methods and even rivaled "source-needed" techniques.

Performance comparison on the Office-31 dataset, showing $A^2Net$ outperforming both source-free and source-needed methods.

On the Office-31 dataset, $A^2Net$ achieved an average accuracy of 90.1%, surpassing the popular SHOT method (88.6%). On the more complex Office-Home dataset, it reached 72.8% average accuracy. Most notably, on the large-scale VisDA synthetic-to-real challenge, it scored 84.3%, a substantial improvement over the source-only baseline of 52.4%. These results suggest that modeling the internal structure of the target domain can be as effective as, or even superior to, explicit source-target alignment.

Conclusion

$A^2Net$ demonstrates that effective domain adaptation is possible without source data by leveraging adversarial classifier discrepancy and self-supervised structural learning. While it relies on a reasonably competent initial source model and involves iterative optimization, its performance across diverse benchmarks establishes it as a robust solution for privacy-preserving and resource-constrained transfer learning. Future work may explore more sophisticated self-supervised tasks to further enhance feature robustness.

Summary updated

Abstract

Unsupervised Domain Adaptation solves knowledge transfer along with the coexistence of well-annotated source domain and unlabeled target instances. However, the source domain in many practical applications is not always accessible due to data privacy or the insufficient memory storage for small devices. This scenario defined as Source-free Domain Adaptation only allows accessing the well-trained source model for target learning. To address the challenge of source data unavailability, we develop an Adaptive Adversarial Network (A 2 Net) including three components. Specifically, the first one named Adaptive Adversarial Inference seeks a target-specific classifier to advance the recognition of samples which the provided source-specific classifier difficultly identifies. Then, the Contrastive Category-wise Matching module exploits the positive relation of every two target images to enforce the compactness of subspace for each category. Thirdly, Self-Supervised Rotation facilitates the model to learn additional semantics from target images by themselves. Extensive experiments on the popular cross-domain benchmarks verify the effectiveness of our proposed model on solving adaptation task without any source data.
BibTeX
@inproceedings{xia2021adaptive,
  author = {Haifeng Xia and Handong Zhao and Zhengming Ding},
  title = {{Adaptive Adversarial Network for Source-free Domain Adaptation}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2021},
  doi = {10.1109/iccv48922.2021.00888},
}
Figure