Abstract
Unsupervised Domain Adaptation (UDA) aims to transfer knowledge from a labeled source domain to an unlabeled target domain. Theoretical bounds for UDA typically involve three terms: source risk, distribution discrepancy, and a combined risk term ($\lambda$). While most existing methods focus on minimizing the first two, they often assume $\lambda$ is a negligible constant. This paper challenges this assumption, demonstrating that combined risk fluctuates significantly during training and can impede performance. We propose E-MixNet, a novel framework that actively minimizes combined risk using an "enhanced mixup" (e-mixup) strategy and a robust proxy loss, achieving state-of-the-art results across multiple benchmarks.
Introduction
The core challenge of Unsupervised Domain Adaptation (UDA) is bridging the gap between a labeled source dataset and an unlabeled target dataset. Theoretical analysis suggests that the target error is bounded by the source error, the discrepancy between domain distributions, and the "ideal" joint error, known as the combined risk ($\lambda$). Historically, researchers have treated $\lambda$ as a fixed, small constant. However, in deep learning, the feature representation is learned rather than fixed. As the feature space evolves, the combined risk can increase, particularly if the model maps different classes from different domains to the same feature space regions.
Rethinking Combined Risk
The authors prove that combined risk is not a constant but a dynamic variable tied to the conditional distribution discrepancy. If a feature transformation collapses distinct class boundaries across domains, the "ideal" shared classifier will inevitably fail, causing $\lambda$ to spike.
As shown in Figure 3, methods that ignore combined risk (like standard pseudo-labeling or mixup) often see it rise as training progresses, which correlates with plateauing or declining accuracy. E-MixNet (e-mixup) maintains a lower and more stable combined risk throughout the training process.
The E-MixNet Framework
To address this, the authors introduce E-MixNet, designed to explicitly minimize the combined risk proxy.
Enhanced Mixup (e-mixup)
Minimizing combined risk requires labels for both domains. Since target labels are unavailable, E-MixNet utilizes pseudo-labels. To improve robustness against noisy pseudo-labels, the authors propose e-mixup. Unlike standard mixup, e-mixup specifically selects pairs of samples that share the same (pseudo) label but originate from different domains or distant feature regions. This encourages the model to learn a domain-invariant representation that respects class boundaries.
Proxy Loss and Architecture
The framework employs a dedicated classifier ($C^*$) to estimate the combined risk. Instead of standard cross-entropy, the authors use Mean Squared Error (MSE) for the combined risk proxy. MSE is bounded and provides smoother gradients, making it less sensitive to the incorrect pseudo-labels common in early training stages.
Experimental Results
E-MixNet was evaluated on three major UDA benchmarks: Office-31, Image-CLEF, and Office-Home.
- Office-31: Achieved an average accuracy of 90.2%, significantly outperforming the MDD baseline, particularly on the challenging DSLR $\to$ Amazon task.
- Image-CLEF: Set a new state-of-the-art with 91.0% average accuracy.
- Office-Home: On this complex 65-class dataset, E-MixNet reached 70.6%, a notable improvement over the previous best of 68.1%.
The authors also demonstrated that their combined risk proxy is "plug-and-play." Integrating it into existing architectures like DANN or CDAN consistently yielded performance gains, validating the theoretical importance of $\lambda$.
Conclusion and Future Work
This research highlights a critical oversight in UDA: the non-constant nature of combined risk. By introducing E-MixNet and the e-mixup strategy, the authors provide a practical way to monitor and minimize this risk. While the method is sensitive to initial pseudo-label quality, it represents a significant step toward more theoretically grounded and high-performing domain adaptation. Future work may explore self-supervised pre-training to further stabilize the initial pseudo-labeling phase.
Summary updated