LEAD: Learning Decomposition for Source-free Universal Domain Adaptation

Sanqing Qu, Tianpei Zou, Lianghua He, Florian Röhrbein, Alois Knoll, Guang Chen, Changjun Jiang

Introduction

Deep learning models frequently encounter performance degradation when deployed in target domains that differ from their source training data. This challenge is compounded in Universal Domain Adaptation (UniDA), where the target domain may contain entirely new "private" categories or lack certain source categories. With increasing privacy concerns, Source-free UniDA (SF-UniDA) has emerged, requiring adaptation without access to original source data. The primary hurdle in SF-UniDA is "knowing what you don't know": distinguishing between known classes under covariate shift and entirely new, unknown categories.

Methodology: Learning Decomposition (LEAD)

The proposed LEArning Decomposition (LEAD) framework addresses SF-UniDA through a geometric lens rather than relying on computationally expensive iterative clustering or brittle hand-crafted thresholds.

Overview of the LEAD framework, illustrating the transition from traditional to source-free UniDA and the internal feature decomposition process.

Orthogonal Feature Decomposition

LEAD leverages the observation that a source classifier's weights ($W_{cls}$) define a "source-known" subspace. Using Singular Value Decomposition (SVD), the authors identify the orthogonal complement—the "source-unknown" space. Target-private (unknown) samples are hypothesized to project more strongly into this unknown space than common samples.

Identification and Adaptation

  1. Unknown Indicator: A two-component Gaussian Mixture Model (GMM) is applied to these projections to model the distribution and separate common classes from private ones.
  2. Adaptive Decision Boundaries: Instead of static global thresholds, LEAD employs instance-level adaptive boundaries based on a "common score," calculated using distances to target prototypes and source anchors. This allows flexibility across categories with varying levels of domain shift.
  3. Optimization: The model is optimized using cross-entropy on pseudo-labels, alongside feature decomposition and consensus regularizers to sharpen class distinctions and ensure local feature consistency.

Experimental Evaluation

LEAD was tested on major benchmarks including Office-31, Office-Home, VisDA, and DomainNet across various scenarios like Open-Partial (OPDA), Open-Set (OSDA), and Partial-Set (PDA).

Key Results and Insights

LEAD demonstrates significant improvements in both accuracy and efficiency. In the challenging VisDA OPDA scenario, it outperformed the previous state-of-the-art (GLC) by 3.5% in H-score (a metric balancing accuracy on common and private classes).

H-score performance on Office-Home (Ar -> Pr and Re -> Cl) showing LEAD's superiority and its ability to boost existing methods like UMAD and GLC across varying numbers of unknown classes.

Efficiency and Versatility

A standout feature of LEAD is its computational efficiency. On the massive DomainNet dataset, LEAD determined decision boundaries in just 0.29 seconds, a reduction of over 99.9% compared to the clustering-based GLC (897.65 seconds). Furthermore, LEAD acts as a "plug-and-play" module; integrating it with the UMAD framework on Office-Home increased the H-score from 70.1% to 78.0%, demonstrating its robustness as a refined unknown detector.

Conclusion

By utilizing the "null space" of source classifier weights, LEAD provides an elegant and highly efficient solution for identifying unknown categories in source-free environments. Its ability to complement existing frameworks while bypassing the curse of dimensionality makes it a powerful tool for privacy-preserving domain adaptation.

Summary updated

Abstract

Universal Domain Adaptation (UniDA) targets knowledge transfer in the presence of both covariate and label shifts. Recently, Source-free Universal Domain Adaptation (SF-UniDA) has emerged to achieve UniDA without access to source data, which tends to be more practical due to data protection policies. The main challenge lies in determining whether covariate-shifted samples belong to target-private unknown categories. Existing methods tackle this either through hand-crafted thresholding or by developing time-consuming iterative clustering strategies. In this paper, we propose a new idea of LEArning Decomposition (LEAD), which decouples features into source-known and-unknown components to identify target-private data. Technically, LEAD initially leverages the or-thogonal decomposition analysis for feature decomposition. Then, LEAD builds instance-level decision boundaries to adaptively identify target-private data. Extensive experiments across various UniDA scenarios have demonstrated the effectiveness and superiority of LEAD. Notably, in the OPDA scenario on VisDA dataset, LEAD outperforms GLC by 3.5% overall H-score and reduces 75% time to derive pseudo-labeling decision boundaries. Besides, LEAD is also appealing in that it is complementary to most existing methods. The code is available at https://github.com/ispc-lab/LEAD.
BibTeX
@inproceedings{qu2024lead,
  author = {Sanqing Qu and Tianpei Zou and Lianghua He and Florian Röhrbein and Alois Knoll and Guang Chen and Changjun Jiang},
  title = {{LEAD: Learning Decomposition for Source-free Universal Domain Adaptation}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2024},
}
Figure