Abstract: Unsupervised domain adaptation (UDA) addresses the challenge of classifying unlabeled data in a target domain using labeled data from a related source domain with a different distribution. While most recent methods focus on explicitly aligning feature distributions, this paper introduces DisClusterDA, which re-casts the problem as discriminative clustering of target data. By leveraging labeled source data as privileged information, DisClusterDA employs a soft Fisher-like criterion, adaptive entropy minimization, and centroid classification to learn compact, well-separated feature distributions. Experimental results across five major benchmarks demonstrate that DisClusterDA outperforms existing state-of-the-art methods, suggesting that implicit alignment through clustering can be more effective than explicit distribution matching.
Introduction
In machine learning, "domain shift" remains a significant hurdle: a model trained on high-quality studio images (source domain) often fails when applied to lower-quality real-world data (target domain). Traditional Unsupervised Domain Adaptation (UDA) attempts to solve this by forcing the feature distributions of both domains to look identical. However, this explicit alignment can be a double-edged sword; if label distributions differ, forcing alignment can distort the underlying data structure, "smearing" distinct classes together in the feature space.
DisClusterDA proposes a shift in perspective: instead of obsessing over alignment, the model should focus on clustering the target data while using the source data as a guide. This approach, termed "distilled discriminative clustering," allows for implicit alignment that preserves the discriminative power of the features.
The DisClusterDA Framework
The core of DisClusterDA is built upon three technical pillars designed to refine the target domain's feature space:
- Adaptive Filtering Entropy Minimization (AFEM): Standard entropy minimization can be sensitive to noise. AFEM introduces an exponential weighting scheme that de-prioritizes low-confidence samples early in training, preventing the model from over-fitting to incorrect pseudo-labels.
- Soft Fisher-like Criterion: This objective encourages intra-cluster compactness (keeping samples of the same class close) and inter-cluster separation (pushing different classes apart), borrowing from classical statistical principles to structure the feature space.
- Centroid Classification: To ensure that target clusters correspond to the correct source classes, the model learns to classify the cluster centroids. This maintains a consistent class ordering across both domains and acts as a geometric regularizer.
By training the network jointly on labeled source data and these target clustering objectives, the source data acts as a "teacher," distilling discriminative knowledge into a shared feature extractor.
Experimental Evaluation
The authors evaluated DisClusterDA on five popular benchmarks, ranging from small-scale digit recognition to large-scale synthetic-to-real challenges:
- Office-31 & Office-Home: DisClusterDA achieved superior accuracy (90.8% on Office-31 and 71.4% on Office-Home), consistently beating adversarial-based methods.
- VisDA-2017: In this large-scale synthetic-to-real task, the method demonstrated robust performance and stability.
- Multi-Source DA: On the Office-Caltech10 benchmark, the model reached a near-perfect 98.1% average accuracy, showcasing its ability to integrate knowledge from multiple source domains simultaneously.
Discussion and Insights
A key finding of this research is that implicit alignment can be superior to explicit alignment. In the DisClusterDA framework, adding an explicit alignment term (like those used in DANN) actually decreased performance. This suggests that when clustering is sufficiently strong, domains align naturally without the risk of distorting feature structures.
Furthermore, the use of prototypes (centroids) proved more effective than focusing solely on individual samples. Centroids are less susceptible to the "domain gap" and provide a more stable semantic representation of classes. This helps prevent "space folding," a common issue in deep networks where class-wise distributions become fragmented.
Conclusion
DisClusterDA demonstrates that treating domain adaptation as a discriminative clustering problem is a powerful alternative to traditional distribution alignment. While currently limited to closed-set scenarios where source and target domains share the same classes, the method provides a robust foundation for future UDA research. Future work may explore extending this framework to partial or open-set domain adaptation and developing metrics to quantify the preservation of discriminative structures during adaptation.
Summary updated