Contrastive Representation Learning Helps Cross-institutional Knowledge Transfer: A Study in Pediatric Ventilation Management

Yuxuan Liu, Jinpei Han, Padmanabhan Ramnarayan, A. Aldo FaisalarXiv 2025 · DOI · arXiv

Introduction

Machine learning models in healthcare often struggle with a "generalization gap." A model trained at a large academic center might perform exceptionally well in its original environment but fail when deployed at a smaller or specialized hospital. This performance degradation is primarily driven by domain shift—differences in patient demographics, disease prevalence, and local clinical practices. This study explores how self-supervised contrastive learning can bridge this gap, specifically in the high-stakes domain of pediatric mechanical ventilation.

The Challenge of Pediatric Ventilation

Deciding when to extubate (remove the breathing tube) a child is a critical clinical decision. Extubating too late increases the risk of ventilator-associated infections and airway trauma, while extubating too early can lead to respiratory failure and emergency re-intubation. Smaller, specialized units, such as cardiac pediatric intensive care units (PICUs), often lack the massive datasets required to train robust deep-learning models from scratch. The central question of this research is whether we can "borrow" physiological patterns learned from a large, general hospital to assist these specialized units.

Overview of Cross-institutional Knowledge Transfer in Healthcare

Methodology: Contrastive Predictive Coding

The researchers employed Contrastive Representation Learning, specifically an architecture known as Contrastive Predictive Coding (CPC). The core objective of CPC is to learn a data representation by predicting future states. By analyzing sequences of clinical data—such as vital signs and laboratory results—the model learns to distinguish the actual future state of a patient from "negative samples" (randomly selected states from other patients).

Clinical Domain Adaptation

Standard CPC typically uses random negative sampling. However, this study introduced similarity-guided sampling. By utilizing clinical domain knowledge (e.g., "time-to-extubation"), the researchers selected more challenging negative samples. This forces the model to learn nuanced physiological differences rather than simple, obvious distinctions between patients.

Two-Phase Transfer Strategy

The transfer process involves two distinct phases:

  1. Pre-training: The model learns general physiological representations from a large source dataset without requiring explicit labels.
  2. Fine-tuning: The model is adapted to the target hospital’s specific tasks using a smaller labeled dataset. The researchers compared "Full Fine-Tuning" (FTF), where the entire model is updated, against "Decoder-only Fine-Tuning" (FTD), where the pre-trained encoder remains frozen.

Cross-Institutional Data and Domain Shift

The study utilized high-resolution Electronic Healthcare Record (EHR) data from two distinct PICUs (2013–2022):

  • Source Institution: A general PICU with 1,883 ventilation episodes.
  • Target Institution: A cardiac-focused PICU with 1,932 episodes.

These populations exhibited significant domain shift. The target unit was composed of 84.8% cardiac patients (compared to 1.6% in the source), featured younger patients, and required more intensive use of vasoactive medications. This disparity makes direct model transfer highly ineffective.

Key Results and Performance

The researchers evaluated two primary tasks: Point-of-care risk assessment (Task 1) and Prospective window identification (Task 2). The results demonstrated that while direct transfer (Source-Only) fails, the CPC-based approach provides a significant advantage, particularly when data is scarce.

Model Performance on Target Institution Under Different Data Regimes

The Power of Few-Shot Learning

The most striking finding occurred in the "few-shot" scenario (5% training data regime). When the target hospital had only a tiny amount of labeled data, the pre-trained CPC model with full fine-tuning (CPC+FTF) significantly outperformed models trained from scratch (Target-Only). In Task 1, CPC+FTF achieved an AUROC of 0.736, compared to 0.716 for the Target-Only baseline. This suggests that foundation models trained on large datasets can provide a viable path for specialized clinics with limited data resources.

Discussion and Insights

Several critical insights emerged from the study:

  • Fine-Tuning is Mandatory: Frozen representations (FTD) performed poorly compared to full fine-tuning (FTF). Effective knowledge transfer requires the model to adapt its internal feature extraction to the specific nuances of the new population.
  • Asymmetric Transfer: Temporal progression patterns (Task 2) transferred more effectively than point-of-care decisions (Task 1). This suggests that while general physiology is shared across institutions, the final clinical decision to extubate is heavily influenced by local hospital culture and specific risk thresholds.

Conclusion

This research demonstrates that self-supervised contrastive learning is a powerful tool for cross-institutional knowledge transfer in healthcare. By pre-training on large-scale general data and fine-tuning on specialized populations, models can overcome the limitations of small datasets and domain shift. Future work will focus on expanding this validation across more diverse geographic regions and improving the interpretability of the learned representations to build clinical trust.

Summary updated

BibTeX
@misc{liu2025contrastive,
  author = {Yuxuan Liu and Jinpei Han and Padmanabhan Ramnarayan and A. Aldo Faisal},
  title = {{Contrastive Representation Learning Helps Cross-institutional Knowledge Transfer: A Study in Pediatric Ventilation Management}},
  year = {2025},
  eprint = {2501.13587},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2501.13587},
}
Figure