Introduction
Unsupervised Domain Adaptation (UDA) aims to bridge the gap between a labeled source domain and an unlabeled target domain. Success in UDA typically hinges on two competing objectives: Domain Alignment, which seeks to make source and target feature distributions indistinguishable, and Class Discriminability, which ensures that features remain separable for classification.
However, these objectives often exist in a state of tension. Aggressive domain alignment can lead to "feature collapse," where class boundaries are blurred, while prioritizing discriminability can prevent the domains from aligning effectively. Most existing methods rely on static weighting for these losses, failing to adapt to the evolving needs of the model during training.
The DWL Framework
To address this "tug-of-war," the authors propose Dynamic Weighted Learning (DWL). This framework introduces a feedback mechanism that monitors the state of alignment and discriminability in real-time to adjust the training objective dynamically.
1. Dynamic Weighting ($\tau$)
DWL employs a dynamic balance factor, $\tau$, to weight the Domain Alignment Loss ($L_{da}$) against the Class Discrimination Loss ($L_{cd}$).
- Monitoring: The system tracks the Maximum Mean Discrepancy (MMD) to gauge alignment and the Linear Discriminant Analysis (LDA) criterion ($J(W)$) to measure discriminability.
- Feedback Loop: If alignment is lagging (high MMD), $\tau$ increases to prioritize domain matching. If class boundaries begin to blur (low $J(W)$), $\tau$ decreases to emphasize discriminability. This ensures the model maintains an optimal balance throughout the learning process.
2. Universal Sample Weighting
Beyond the loss trade-off, UDA often suffers from sample imbalance between domains. DWL introduces a weighting scheme where samples are weighted inversely to their domain's proportion in the total dataset. This prevents the optimization process from being dominated by the larger domain, ensuring that information from both source and target is treated with equal importance.
Architecture
The DWL architecture consists of a feature generator ($G$), a domain discriminator ($D$), a main classifier ($C$), and two auxiliary classifiers ($C_1, C_2$).
The feature generator $G$ learns representations that are both domain-invariant (guided by $D$) and class-discriminative (guided by $C, C_1, C_2$). The dynamic factor $\tau$ acts as a meta-learner, adjusting the influence of the domain alignment and class discrimination branches on the fly.
Experimental Results
DWL was evaluated on several competitive UDA benchmarks, consistently achieving state-of-the-art results:
- VisDA-2017: Achieved 77.1% average accuracy, outperforming previous methods like BSP.
- Office-31: Reached an average accuracy of 87.1%, showing significant gains in cross-domain tasks like DSLR $\to$ Amazon.
- Digits: Attained a near-perfect 97.6% average accuracy across MNIST, USPS, and SVHN transfers.
Ablation studies confirmed that both the dynamic $\tau$ weighting and the universal sample weighting are critical to these performance gains. Removing either component resulted in a noticeable drop in accuracy, validating the importance of addressing both the objective trade-off and data imbalance.
Conclusion
Dynamic Weighted Learning redefines the UDA training process as a self-correcting system. By treating domain alignment and class discriminability as dynamic variables rather than static constraints, DWL effectively mitigates negative transfer and provides a robust framework for adapting models to new, unlabeled environments.
Summary updated