On Target Shift in Adversarial Domain Adaptation

Yitong Li, Michael Murias, Samantha Major, Geraldine Dawson, David E. Carlson

Abstract

Discrepancy between training and testing domains is a fundamental problem in the generalization of machine learning techniques. Recently, several approaches have been proposed to learn domain invariant features to bridge the gap between domains. However, most existing methods assume that the label distributions are identical across domains, which is often violated in real-world applications. This paper introduces DATS (Domain Adversarial nets for Target Shift), a framework designed to handle target shift by estimating class proportions in the target domain and weighting source data accordingly.

Introduction

Most unsupervised domain adaptation (UDA) techniques are designed to mitigate covariate shift, where input features change but the relationship between features and labels remains constant. These methods typically assume that label proportions (e.g., the ratio of different classes) are consistent across domains. In practice, this assumption often fails—a phenomenon known as target shift or label shift.

Standard adversarial methods, such as Domain Adversarial Neural Networks (DANN), attempt to make features from source and target domains indistinguishable. However, if class balances differ significantly, the domain discriminator can identify the domain based solely on these skewed proportions. To fool the discriminator, the model may misclassify target samples to match the source's distribution, leading to a significant drop in predictive performance.

Methodology: Domain Adversarial Nets for Target Shift (DATS)

To address the "adversarial trap" of label shift, the authors propose the DATS framework. DATS extends the standard adversarial architecture with three key components:

  1. Label Proportion Estimator: DATS estimates target label proportions ($\gamma^T$) by matching marginal distributions in the latent feature space. This is achieved through either Mean Matching (aligning average feature vectors) or Distribution Matching (using kernel methods to match higher-order moments).
  2. Source Domain Weighting: In multi-source scenarios, DATS learns a weighting vector ($\lambda$) to prioritize source domains that are most similar to the target domain based on learned features.
  3. Weighted Adversarial Loss: The estimated label proportions are used to re-weight source samples in the adversarial loss function. This ensures the discriminator focuses on matching conditional distributions rather than being misled by global class imbalances.
The DATS architecture with Label Proportion Estimator.

Experimental Evaluation and Results

The authors evaluated DATS against several benchmarks, including synthetic digit tasks and real-world biological datasets.

Robustness to Imbalance

On the MNIST to MNISTM task, the researchers synthetically varied the proportions of specific digits. While the baseline DANN's performance fluctuated and dropped sharply as the label shift became extreme, DATS maintained high and stable accuracy across the entire spectrum. Specifically, on the MNISTM target task, DATS achieved 68.2% accuracy, significantly outperforming standard DANN (56.6%) and multi-source baselines like MDANs (59.6%).

Comparison of AUC stability under varying label proportions.

Real-World Biological Applications

The framework's utility was further demonstrated on high-stakes biological data:

  • ASD Dataset: In EEG recordings from children with Autism Spectral Disorder, DATS achieved 67.2% accuracy, compared to 63.8% for DANN.
  • LFP Dataset: In Local Field Potential signals from mice, DATS showed similar improvements, proving its robustness in noisy environments where label shift occurs naturally.

Conclusion

DATS addresses a critical blind spot in adversarial domain adaptation by explicitly accounting for target shift. By performing label proportion estimation in the learned feature space rather than the raw input space, the framework provides a robust and computationally feasible solution for real-world UDA tasks. Future work may explore the computational complexity of kernel methods for larger datasets and the theoretical risks of feedback loops in multi-source adaptation.

Summary updated

Abstract

Discrepancy between training and testing domains is a fundamental problem in the generalization of machine learning techniques. Recently, several approaches have been proposed to learn domain invar...
BibTeX
@inproceedings{li2019target,
  author = {Yitong Li and Michael Murias and Samantha Major and Geraldine Dawson and David E. Carlson},
  title = {{On Target Shift in Adversarial Domain Adaptation}},
  booktitle = {International Conference on Artificial Intelligence and Statistics},
  year = {2019},
  url = {https://openreview.net/forum?id=C4RZabFB5P8},
}
Figure