Abstract
Unsupervised domain adaptation methods traditionally assume that all source categories are present in the target domain. In practice, little may be known about the category overlap between the two domains. While some methods address target settings with either partial or open-set categories, they assume that the particular setting is known a priori. We propose a more universally applicable domain adaptation framework that can handle arbitrary category shift, called Domain Adaptative Neighborhood Clustering via Entropy optimization (DANCE). DANCE combines two novel ideas: First, as we cannot fully rely on source categories to learn features discriminative for the target, we propose a novel neighborhood clustering technique to learn the structure of the target domain in a self-supervised way. Second, we use entropy-based feature alignment and rejection to align target features with the source, or reject them as unknown categories based on their entropy. We show through extensive experiments that DANCE outperforms baselines across open-set, open-partial and partial domain adaptation settings. Implementation is available at https://github.com/VisionLearningGroup/DANCE.
Introduction
In Unsupervised Domain Adaptation (UDA), the goal is to transfer knowledge from a labeled "source" domain (e.g., studio photos) to an unlabeled "target" domain (e.g., real-world snapshots). Traditional UDA assumes a closed-set scenario where both domains share identical categories. However, real-world data is often messy, leading to "Universal" Domain Adaptation (UniDA) challenges where category overlap is unknown. The target domain might contain only a subset of source classes (Partial DA), entirely new "unknown" classes (Open-set DA), or a combination of both.
Existing methods often fail when the category shift is unknown because they attempt to force-align target features with source prototypes. This leads to "catastrophic misalignment" when the model tries to force unknown objects into known categories. DANCE addresses this by letting the target domain "speak for itself" through self-supervised clustering and entropy-based rejection.
Methodology: The DANCE Framework
DANCE introduces two core mechanisms to handle arbitrary category shifts without prior knowledge of domain overlap.
Neighborhood Clustering (NC)
Since source labels may not represent all target categories, DANCE employs self-supervision to discover the target domain's inherent structure. The model maintains a "memory bank" of target features. For each target sample, it calculates similarity to other target samples and source prototypes. By minimizing the entropy of this similarity distribution, the model encourages each target point to cluster with its nearest neighbors—regardless of whether those neighbors belong to a known source class or an unknown target category.
Entropy Separation (ES) Loss
The ES loss acts as a gatekeeper to distinguish between known and unknown classes. It leverages the model's prediction entropy: low entropy indicates confidence in a known class, while high entropy suggests an unknown category. DANCE defines an entropy boundary (ρ) and pushes samples away from it. Confident samples are aligned with the source, while ambiguous, high-entropy samples are pushed further into the "unknown" region to be rejected. Additionally, Domain-Specific Batch Normalization is used to account for stylistic differences between domains without forcing hard feature alignment.
Experimental Results
The researchers evaluated DANCE on major benchmarks including Office-31, OfficeHome, and VisDA, simulating various category shift scenarios.
- Universal Superiority: DANCE was the only method to consistently outperform the "Source Only" baseline across all experimental settings (Closed-set, Partial, Open-set, and Open-Partial).
- State-of-the-Art Accuracy: Across all scenarios, DANCE achieved an average accuracy of 77.3%, significantly surpassing previous universal methods like UAN (62.0%) and ETN (70.5%).
- Visual Clustering: t-SNE visualizations demonstrate that DANCE successfully clusters unknown target samples into distinct groups, preventing them from interfering with the alignment of known classes.
Conclusion
DANCE provides a robust, "out-of-the-box" solution for domain adaptation under unknown category shifts. By combining self-supervised neighborhood clustering with entropy-based separation, it avoids the pitfalls of forced alignment. While the method relies on certain hyperparameters like the entropy boundary ρ, its consistent performance across diverse datasets demonstrates its potential for real-world universal domain adaptation.
Summary updated