Unsupervised Domain Expansion from Multiple Sources

Jing Zhang, Wanqing Li, Lu Sheng, Chang Tang, Philip OgunbonaarXiv 2020 · arXiv

Abstract

This paper introduces Unsupervised Multi-Source Domain Expansion (UMSDE), a framework for adapting AI models to new environments without losing their ability to handle original ones—all while having zero access to the original training data. By leveraging the "disagreement" between multiple pre-trained models on unlabelled new data, the authors successfully expand a system's expertise without the risk of catastrophic forgetting.

Introduction

In real-world applications, AI systems often transition from their initial training environments (source domains) to new, unseen environments (target domains). Standard adaptation techniques face significant hurdles: data privacy regulations often prevent the retention of source data, fine-tuning frequently leads to catastrophic forgetting of original tasks, and target data is typically unlabelled. While Unsupervised Domain Adaptation (UDA) addresses target performance, it often neglects source domain retention. Domain Generalization (DG) aims for robustness but requires simultaneous access to all source data. UMSDE bridges this gap by enabling "Domain Expansion"—growing a model's knowledge to encompass both old and new domains using only "black-box" pre-trained models and unlabelled target data.

Methodology

The proposed UMSDE framework treats pre-trained source models as "experts" with distinct biases. The core mechanism utilizes predicted Class Probability Distributions (pCPDs) to align these models on the new domain.

The UMSDE pipeline consists of three stages: pre-training on separate source domains, unsupervised expansion using unlabelled new data, and final fusion for testing across all domains.

The approach incorporates several key components:

  • Domain Alignment Loss ($L_{bias}$): Unlabelled target data acts as a bridge. By forcing different source models to reach a consensus on their predictions for this data, the framework reduces individual source-specific biases.
  • Entropy-based Weighting: Recognizing that not all source models are equally relevant to a new domain, the system calculates the entropy of each model’s predictions. Models exhibiting lower entropy (higher certainty) are deemed more compatible and are assigned higher weights during alignment.
  • Source Information Preservation ($L_{org}$): To prevent catastrophic forgetting, a regularization loss ensures that updated models do not deviate excessively from their original source-specific expertise.
  • Ad-hoc Fusion: During inference, the outputs of the expanded models are fused (e.g., via average or max fusion) to provide a final classification robust across the expanded domain.

Key Insights

A central finding of this work is the use of agreement as a proxy for alignment. By aligning the probability distributions of models trained on different sources over a third, unlabelled domain, internal biases are effectively neutralized without needing the original data.

Empirical evidence shows a strong negative correlation between the entropy of predicted class probability distributions and model accuracy on a new domain.

Furthermore, the paper demonstrates a near-linear negative relationship between prediction entropy and accuracy. This allows the system to self-calibrate, identifying which models are most reliable for the new domain without human intervention. This shift from "adaptation" to "expansion" is particularly valuable for edge computing and robotics, where continuous functionality across diverse environments is critical.

Experimental Evaluation

The UMSDE method was evaluated on three standard benchmarks: VLCS (diverse photographic styles), ImageCLEF_DA (12 shared categories across 4 domains), and PACS (Art, Cartoon, Photo, and Sketch).

The results were compelling:

  • UMSDE outperformed the baseline (simple fusion of original models) in 11 out of 12 scenarios.
  • On the challenging PACS dataset, significant accuracy gains were observed when expanding into "Art" and "Cartoon" domains.
  • Crucially, the models maintained or even improved performance on original source domains while gaining proficiency in the new target domain.
  • The framework proved robust across various architectures, including AlexNet and ResNet50.

Conclusion and Future Work

UMSDE provides a practical path for domain expansion under strict data privacy and labeling constraints. Future research will address current limitations, such as the assumption of a shared label space (Open Set scenarios) and the potential for more sophisticated model fusion techniques beyond simple averaging. Additionally, automating the selection of hyperparameters like softmax temperature ($T$) remains an area for further optimization.

Summary updated

Abstract

Given an existing system learned from previous source domains, it is desirable to adapt the system to new domains without accessing and forgetting all the previous domains in some applications. This problem is known as domain expansion. Unlike traditional domain adaptation in which the target domain is the domain defined by new data, in domain expansion the target domain is formed jointly by the source domains and the new domain (hence, domain expansion) and the label function to be learned must work for the expanded domain. Specifically, this paper presents a method for unsupervised multi-source domain expansion (UMSDE) where only the pre-learned models of the source domains and unlabelled new domain data are available. We propose to use the predicted class probability of the unlabelled data in the new domain produced by different source models to jointly mitigate the biases among domains, exploit the discriminative information in the new domain, and preserve the performance in the source domains. Experimental results on the VLCS, ImageCLEF_DA and PACS datasets have verified the effectiveness of the proposed method.
BibTeX
@misc{zhang2020unsupervised,
  author = {Jing Zhang and Wanqing Li and Lu Sheng and Chang Tang and Philip Ogunbona},
  title = {{Unsupervised Domain Expansion from Multiple Sources}},
  year = {2020},
  eprint = {2005.12544},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2005.12544},
}
Figure