Multiple Source Domain Adaptation with Adversarial Learning

Han Zhao, Shanghang Zhang, Guanhang Wu, João Paulo Costeira, José M. F. Moura, Geoffrey J. Gordon

TL;DR

This paper introduces Multisource Domain Adversarial Networks (MDANs), a deep learning framework designed to adapt a model to an unlabeled target domain using data from multiple labeled source domains. By deriving a new generalization bound and using adversarial training, the authors learn feature representations that are simultaneously discriminative for the main task and invariant across all source and target distributions.

The Challenge of Multiple Sources

In real-world machine learning, data rarely originates from a single, clean distribution. Domain Adaptation (DA) helps models trained on a "source" domain (like synthetic images) perform well on a "target" domain (like real photos). However, most existing research focuses heavily on the single-source-single-target setting.

In practice, we often have access to multiple distinct datasets (e.g., sentiment data from books, electronics, and movies). Naively combining these sources into one massive training set often leads to suboptimal results because each source has its own unique "shift" relative to the target. Previous theoretical bounds for multi-source DA often required knowing the target distribution or the "optimal" way to combine sources beforehand—information that is rarely available.

Multisource Domain Adversarial Networks (MDANs)

To address this, the authors derive a novel generalization bound for multi-source DA that does not require expert knowledge of the target distribution. Based on this theory, they propose two versions of MDANs:

  1. Hard-Max MDAN: This model directly optimizes the theoretical bound by minimizing the maximum discrepancy between the target and any individual source domain. It relies on a minimax game where a feature extractor tries to "fool" several domain classifiers (one for each source-target pair).
  2. Soft-Max MDAN: Recognizing that the "Hard-Max" version can be data-inefficient (as it only learns from the "worst" source at each training step), the authors propose a smoothed version using a log-sum-exp approximation. This allows the model to adaptively weight all source domains during training, focusing more heavily on the sources that are currently hardest to align with the target.
Architecture of the Multisource Domain Adversarial Network (MDAN), illustrating the feature extractor, task classifier, and multiple domain classifiers optimized via gradient reversal.

Both models utilize a Gradient Reversal Layer (GRL), a standard technique in adversarial DA. The GRL allows the network to learn features that the domain classifiers cannot distinguish, effectively "erasing" domain-specific signatures while keeping the features useful for the primary task.

Experimental Validation

The authors evaluated MDANs across three distinct types of data, consistently outperforming state-of-the-art baselines like DANN (Domain-Adversarial Neural Networks) and mSDA (Marginalized Stacked Denoising Autoencoders).

  • Sentiment Analysis (Amazon Reviews): An NLP task classifying sentiment across four product categories (Books, DVDs, Electronics, Kitchen). Soft-Max MDAN achieved the highest accuracy in nearly all transfer tasks, proving more robust than simply combining all sources into one baseline model.
  • Digit Classification: A computer vision benchmark combining four popular datasets: MNIST, MNIST-M, SVHN, and SynthDigits. On the notoriously difficult SVHN-to-others transfer, MDANs significantly outperformed single-source models. For example, in the Sv+Mm+Sy -> Mt task, MDAN-Soft-Max reached 97.9% accuracy, approaching the "Target-only" upper bound of 98.7%.
  • Vehicle Counting (WebCamT): A large-scale, real-world dataset for vehicle counting using low-resolution city camera feeds, challenged by varying camera angles, lighting, and weather. The model achieved a Mean Absolute Error (MAE) as low as 1.19, meaning it was off by only about one car per frame on average, despite never seeing a single label from the target camera.

Key Insights

  • The "Worst-Case" Bottleneck: The underlying theory suggests that target performance is limited by the source domain that is least like the target. MDANs are explicitly designed to pull that "worst" source into alignment.
  • Adaptive Weighting: The Soft-Max MDAN acts as an automated ensemble. It dynamically identifies which sources provide the most useful (or most difficult) information for the target task at any given moment, rather than treating all sources equally.
  • More Data $\neq$ Better Performance: A fascinating insight from the vehicle counting experiments showed that adding more source domains can actually hurt performance if those new sources have a very high divergence from the target. This validates the theoretical claim that naively incorporating more source domains is not always beneficial.

Limitations and Future Work

  • The $\lambda$ Factor: The authors note that if the "optimal combined risk" ($\lambda$)—the best possible error a hypothesis can achieve on both the source and target—is high, no amount of adaptation will yield good results. This remains a fundamental limit of DA.
  • Computational Overhead: MDANs require a separate domain classifier for every source domain. While these sub-networks are usually small, the complexity scales linearly with the number of sources.
  • Semi-Supervised Extension: The current framework assumes the target domain is entirely unlabeled. Incorporating even a small amount of labeled target data (semi-supervised DA) could further tighten the performance gap in future iterations.

Summary updated

Abstract

While domain adaptation has been actively researched in recent years, most theoretical results and algorithms focus on the single-source-single-target adaptation setting. Naive application of such algorithms on multiple source domain adaptation problem may lead to suboptimal solutions. We propose a new generalization bound for domain adaptation when there are multiple source domains with labeled instances and one target domain with unlabeled instances. Compared with existing bounds, the new bound does not require expert knowledge about the target distribution, nor the optimal combination rule for multisource domains. Interestingly, our theory also leads to an efficient learning strategy using adversarial neural networks: we show how to interpret it as learning feature representations that are invariant to the multiple domain shifts while still being discriminative for the learning task. To this end, we propose two models, both of which we call multisource domain adversarial networks (MDANs): the first model optimizes directly our bound, while the second model is a smoothed approximation of the first one, leading to a more data-efficient and task-adaptive model. The optimization tasks of both models are minimax saddle point problems that can be optimized by adversarial training. To demonstrate the effectiveness of MDANs, we conduct extensive experiments showing superior adaptation performance on three real-world datasets: sentiment analysis, digit classification, and vehicle counting.
BibTeX
@inproceedings{zhao2018multiple,
  author = {Han Zhao and Shanghang Zhang and Guanhang Wu and João Paulo Costeira and José M. F. Moura and Geoffrey J. Gordon},
  title = {{Multiple Source Domain Adaptation with Adversarial Learning}},
  booktitle = {Workshops @ ICLR},
  year = {2018},
  url = {https://openreview.net/forum?id=DSR_zxShHwj},
}
Figure