Abstract
Domain adaptation (DA) addresses distribution shifts by generalizing models from a source to a target domain. Traditional DA assumes shared feature and label spaces, which is often unrealistic in complex real-world applications. This paper introduces Open-Set Heterogeneous Domain Adaptation (OSHeDA), a scenario where both feature spaces and label sets differ between domains. We propose Representation Learning for OSHeDA (RL-OSHeDA), a method guided by a new theoretical framework that simultaneously aligns known categories and segregates unknown samples. Experiments across diverse datasets, including clinical ECG records, demonstrate the superiority of RL-OSHeDA over existing baselines.
The OSHeDA Challenge
Most DA research focuses on scenarios with identical features and categories. However, real-world applications often face two simultaneous hurdles:
- Heterogeneous Features: Data types differ between domains (e.g., digital records vs. scanned paper records).
- Open-Set Labels: The target domain contains novel classes or abnormalities not present during the initial training phase.
Existing techniques like Heterogeneous DA (HeDA) or Open-Set DA (OSDA) typically address only one of these issues. OSHeDA tackles both, preventing "negative transfer" where unknown target samples are incorrectly forced into known source categories, thereby improving model reliability in dynamic environments.
Methodology: RL-OSHeDA
The proposed RL-OSHeDA framework maps source and target data into a shared representation space where a unified classifier can operate. The model is optimized using a four-part objective function derived directly from theoretical learning bounds for target domain prediction error.
Core Components
- Theoretical Alignment: The objective function includes $L_{inv}$ (Maximum Mean Discrepancy) to pull known classes together across domains and $L_{seg}$ to actively push unknown target representations away from known ones.
- Risk Estimation: $L_{osd}$ serves as a non-negative risk estimator to manage the open-set difference, while $L_{cls}$ handles standard classification on labeled data.
- Two-Stage Training: A "warm-up" stage establishes a baseline on labeled data before introducing complex alignment and segregation losses. This prevents noisy initial pseudo-labels from degrading the model's feature mapping.
Experimental Evaluation
The authors evaluated RL-OSHeDA across 56 tasks using seven datasets, including image, text, and clinical data. A standout benchmark was the PTB-XL dataset, which involves adapting digital ECGs to paper records—a direct application of the OSHeDA problem in healthcare.
Key Results
Performance was measured using the Harmonic Mean of Open-Set (HOS), which balances accuracy on known classes with the ability to correctly identify unknown samples.
- Consistent Superiority: RL-OSHeDA outperformed specialized HeDA and OSDA baselines across all 7 datasets.
- Clinical Impact: On the complex PTB-XL task, RL-OSHeDA achieved an HOS of 47.48, significantly higher than the next best baseline (DS3L at 30.30).
- Ablation Success: Removing any component (alignment, segregation, or two-stage training) led to a drop in performance, validating the holistic design of the algorithm.
Conclusion
RL-OSHeDA provides a robust solution for complex domain adaptation tasks where both data format and category sets shift. By combining a rigorous theoretical framework with a "push-pull" dynamic in the representation space, the method effectively bridges the gap between heterogeneous data sources while maintaining sensitivity to novel classes. Future work may explore more advanced unknown detection mechanisms to further refine the segregation of open-set samples.
Summary updated