Rethinking Correlation Learning via Label Prior for Open Set Domain Adaptation

Zixian Huang, Chuan-Xian RenIJCAI 2024 · OpenReview

Abstract

Open Set Domain Adaptation (OSDA) aims to transfer knowledge from a labeled source domain to an unlabeled target domain, where known classes exist across domains while unknown classes are present only in the target domain. Existing methods rely on the clustering structure to identify the unknown classes, which empirically induces a large identification error if the unknown classes are a mixture of multiple components. To break through this barrier, we formulate OSDA from the view of correlation and propose a correlation metric-based framework called Balanced Correlation Learning (BCL). BCL employs Hilbert-Schmidt Independence Criterion (HSIC) to characterize the separation between unknown and known classes, where HSIC is reformulated as the nodes’ relation on graph. By considering the label prior as variable, theoretical results are derived to analytically show a sufficient condition for desired learning direction for OSDA. Methodologically, the class-balanced HSIC is proposed to preserve domain-invariant and class-discriminative features. With the guarantee of correlation learning, the entropy-based principle can effectively identify the unknown classes via uncertainty. Empirically, extensive evaluations are conducted, where BCL achieves significant performance improvements.

Introduction

In the evolving landscape of machine learning, Open Set Domain Adaptation (OSDA) has emerged as a critical challenge. Unlike standard domain adaptation, which assumes a shared label space between source and target domains, OSDA acknowledges the reality of "unknown" classes in the target domain that were never encountered during training. The goal is twofold: accurately aligning known classes across domains and robustly isolating unknown samples to prevent them from being misclassified into known categories.

Problem Statement: The Clustering Barrier

Traditional OSDA approaches often rely on the assumption that unknown classes form a single, cohesive cluster. However, in practice, unknown samples frequently consist of multiple disparate components. When these "unknowns" exhibit a complex, multi-cluster structure, standard identification methods suffer from high error rates.

Figure 1: Comparison between single and multiple cluster structures for unknown classes.

As illustrated in Figure 1, if the unknown classes are not a single cluster, they can easily be misidentified or interfere with the alignment of known classes. Existing methods often fail to recognize these complex structures, leading to significant identification errors.

The Balanced Correlation Learning (BCL) Framework

To address these limitations, this paper introduces Balanced Correlation Learning (BCL). Instead of relying on geometric clustering, BCL utilizes the Hilbert-Schmidt Independence Criterion (HSIC) to measure and optimize the correlation between features and labels.

HSIC as Graph Embedding

The authors provide a mathematical reformulation of HSIC as a graph adjacency matrix. This theoretical bridge allows for a precise analysis of how class proportions—the "label prior"—influence the learning direction. They demonstrate that without accounting for these priors, standard correlation learning can inadvertently reduce the discriminative power of the model in imbalanced scenarios.

Class-Balanced HSIC ($HSIC_B$) and Scatter

A core innovation of BCL is the Class-Balanced HSIC ($HSIC_B$). By incorporating a reweighting scheme based on label priors, BCL ensures that samples from different classes are consistently pushed apart in the feature space, regardless of their relative frequency.

Figure 2: The BCL mechanism for balancing transferability and discriminability.

Furthermore, BCL introduces a "Class Conditional HSIC" ($L_C$) that allows for controlled intra-class scatter. As shown in Figure 2, allowing some variance within a class helps the model mix source and target samples more effectively, leading to more robust decision boundaries that generalize better to the target domain.

Threshold-Free Identification

Moving away from manually tuned "magic thresholds," BCL employs an entropy-based clustering principle. By applying a simple clustering algorithm to the entropy of target samples, the model automatically categorizes data into Low, Medium, and High uncertainty groups. The High uncertainty group is then identified as the unknown class, making the system adaptive to the specific difficulty of the task at hand.

Experimental Results

The BCL framework was rigorously tested on three major benchmarks: Office-31, Office-Home, and VisDA. The primary evaluation metric was the HOS (harmonic mean of accuracy for known and unknown classes).

  • Office-31: BCL achieved a 92.1% HOS, a 3.5% improvement over the previous state-of-the-art (ANNA).
  • Office-Home: The model reached 70.8% HOS, demonstrating strong performance even with 65 categories.
  • VisDA: On this large-scale synthetic-to-real task, BCL achieved 74.1% HOS, outperforming previous methods.
  • Unknown Recognition: The method showed exceptional performance in identifying unknown classes, reaching 94.8% UnK accuracy on specific Office-31 tasks.

Conclusion

By rethinking OSDA through the lens of correlation and label priors, BCL provides a robust, theoretically grounded alternative to clustering-based methods. Its ability to handle complex unknown structures and its threshold-free identification mechanism set a new standard for performance and reliability in open-set scenarios.

Summary updated

Abstract

Open Set Domain Adaptation (OSDA) aims to transfer knowledge from a labeled source domain to an unlabeled target domain, where known classes exist across domains while unknown classes are present only in the target domain. Existing methods rely on the clustering structure to identify the unknown classes, which empirically induces a large identification error if the unknown classes are a mixture of multiple components. To break through this barrier, we formulate OSDA from the view of correlation and propose a correlation metric-based framework called Balanced Correlation Learning (BCL). BCL employs Hilbert-Schmidt Independence Criterion (HSIC) to characterize the separation between unknown and known classes, where HSIC is reformulated as the nodes’ relation on graph. By considering the label prior as variable, theoretical results are derived to analytically show a sufficient condition for desired learning direction for OSDA. Methodologically, the class-balanced HSIC is proposed to preserve domain-invariant and class-discriminative features. With the guarantee of correlation learning, the entropy-based principle can effectively identify the unknown classes via uncertainty. Empirically, extensive evaluations are conducted, where BCL achieves significant performance improvements.
BibTeX
@inproceedings{huang2024rethinking,
  author = {Zixian Huang and Chuan-Xian Ren},
  title = {{Rethinking Correlation Learning via Label Prior for Open Set Domain Adaptation}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2024},
  url = {https://openreview.net/forum?id=PNwJd3lTko},
}
Figure