Introduction
Transferring knowledge between domains is a cornerstone of modern machine learning. However, real-world applications often present two simultaneous challenges: heterogeneous features (where source and target data have different dimensions or modalities) and open-set labels (where the target domain contains novel categories not present during training). This paper, presented at AAAI-25, introduces the Open-Set Heterogeneous Domain Adaptation (OSHeDA) framework to address these combined hurdles.
The OSHeDA Problem
Standard Domain Adaptation (DA) typically assumes either shared features (Open-Set DA) or shared label sets (Heterogeneous DA). In many practical scenarios, such as medical diagnostics, both assumptions fail. For instance, a model trained on digital Electrocardiogram (ECG) signals may need to be deployed in a setting using paper-scanned ECGs, where the clinic also encounters rare heart conditions absent from the original digital training set.
Attempting to align these domains using traditional methods often leads to "negative transfer," where the model mistakenly forces novel target classes into known source categories, degrading overall performance.
The RL-OSHeDA Framework
To solve this, the authors propose Representation Learning for OSHeDA (RL-OSHeDA). The method maps source and target data into a shared representation space designed to align known classes while segregating unknown ones.
Technical Components
The framework is built upon a new theoretical learning bound and utilizes a multi-part loss function:
- Alignment ($L_{inv}$): Employs Maximum Mean Discrepancy (MMD) to ensure that shared classes (e.g., "Normal" rhythms) are mapped to similar regions in the representation space regardless of the input domain.
- Segregation ($L_{seg}$): Explicitly pushes representations of unknown classes away from the centroids of known classes to prevent "smearing."
- Open-Set Difference ($L_{osd}$): A non-negative risk estimator that helps the model identify "unknown" samples without explicit supervision for those classes.
Two-Stage Learning Strategy
A key innovation is the two-stage training process. In Stage 1, the model focuses on classifying known labels to establish stable feature extractors. In Stage 2, it introduces the complex alignment and segregation losses. This prevents the model from being misled by noisy pseudo-labels for unknown classes early in the training process.
Experimental Evaluation
The researchers evaluated RL-OSHeDA across seven diverse benchmarks, including clinical ECG data (PTB-XL), image datasets (CIFAR, ImageNet, Office-Caltech), and text collections (Wikipedia, Reuters).
The primary performance metric was the Harmonic mean of Open-set score (HOS), which balances accuracy on known classes with the ability to detect unknown ones. RL-OSHeDA consistently outperformed state-of-the-art baselines (such as SCT, SSAN, and STN) across all datasets. On average, RL-OSHeDA achieved an HOS of 67.21%, a significant improvement over the next best baseline, SCT (59.89%).
Visualization using t-SNE plots confirmed that RL-OSHeDA creates tighter, more distinct clusters for known classes while maintaining clear separation for unknown instances, whereas baseline methods often resulted in overlapping clusters.
Conclusion and Future Work
RL-OSHeDA provides a robust theoretical and algorithmic foundation for complex domain adaptation tasks. While the current semi-supervised approach requires a small number of labeled target instances, it sets the stage for future research into fully unsupervised OSHeDA. Additionally, investigating the model's sensitivity to the estimated proportion of unknown classes remains an important area for further study.
Summary updated