Abstract
This paper addresses a specific challenge in Unsupervised Domain Adaptation (UDA) where the target domain contains "extra" features not present in the source domain. We propose an iterative algorithm leveraging Optimal Transport (OT) to bridge this dimensionality gap. By simplifying a complex two-way OT problem into a practical one-way mapping, the method generates pseudo-labels for the target domain, allowing a model to learn from both common and extra features. Experimental results on synthetic and real-world sensor datasets demonstrate significant accuracy improvements over existing methods like Joint Distribution Optimal Transport (JDOT), highlighting the method's ability to effectively utilize new information in the target domain without requiring additional labels.
Introduction
In traditional machine learning, models are typically trained and tested on data sharing the same feature space. Domain Adaptation (DA) addresses scenarios where the distribution of these features shifts between the source (training) and target (test) domains. However, a more complex challenge arises in Heterogeneous Domain Adaptation, specifically "Hybrid" DA, where the source and target share common features, but the target domain possesses additional, "extra" features.
A common example is upgrading a sensor suite: a model trained on a basic set of sensors must now operate on a platform equipped with advanced, additional sensors. The core challenge lies in training a model to utilize these new features when the labeled source data lacks them. Existing methods often ignore these extra features or use complex embeddings that obscure the physical meaning of the shared features. This paper introduces a novel OT-based approach to solve this problem while preserving feature homogeneity.
Methodology
The authors propose an iterative algorithm grounded in the mathematical framework of Optimal Transport (OT), which finds the most efficient mapping between probability distributions.
Problem Formulation and the "One-Way" Shortcut
Theoretically, the problem is framed as a "two-way" OT problem: estimating the appearance of extra features in the source domain while simultaneously transferring labels to the target. However, the authors prove that if the relationship between features and labels remains stable across domains, this can be simplified into a practical one-way OT problem from the source to the target.
Iterative Optimization Process
The algorithm proceeds through three main steps:
- Step A (OT Mapping): Solve an OT problem to map source instances to target instances. This mapping is based on the distance between common features and the current model's prediction error.
- Step B (Pseudo-labeling): Use the resulting mapping to assign "pseudo-labels" to the target data points.
- Step C (Model Training): Train a new model in the target domain using the full feature set (common + extra) and the generated pseudo-labels.
Theoretical Learning Bound
Unlike previous works relying solely on Wasserstein distance, the authors derive a tighter learning bound using Rademacher complexity. This provides a more intuitive understanding of how model complexity and the quality of label transfer influence final performance.
Experimental Evaluation
The approach was validated across three distinct data types:
- Synthetic Data: A 2D classification problem (1 common, 1 extra feature) used to visualize handling of overlapping distributions. The proposed method achieved 95.5% accuracy, significantly outperforming JDOT (80.3%), which ignores extra features.
- Gas Sensor Array Drift Dataset: This real-world dataset contains measurements from 16 sensors. By designating 8 as "common" and 8 as "extra," the authors simulated feature mismatch. In a transfer task from Domain 1 to Domain 3, the method reached 96.02% accuracy, compared to a baseline's 52.28%.
- REALDISP Activity Recognition Dataset: This dataset involves physical activity tracking with sensor units. It represents a realistic scenario where sensor misplacement causes both domain shift and feature additions.
Key Insights
Preserving Feature Meaning
By focusing on the distance between common features rather than projecting data into a latent space, the method maintains the physical interpretability of the data. This is particularly vital for sensor-based applications where feature meaning is tied to physical hardware.
The Role of Extra Features
A critical finding is that the OT mapping allows the target model to discover correlations between new features and transferred labels, even though the source labels have no prior "knowledge" of these extra features. Furthermore, the trained model often achieves higher accuracy than the raw OT label transfer, suggesting the model successfully "denoises" pseudo-labels by learning the underlying structure of the extra features.
Limitations and Future Work
The algorithm currently assumes similar class proportions in both domains, which may not hold in "in-the-wild" scenarios. Additionally, highly flexible models like deep neural networks may overfit to early, noisy pseudo-labels, necessitating regularization or early stopping. Future research may explore the inverse problem: adaptation when features present in the source domain become unobservable in the target domain due to sensor failure.
Summary updated