Introduction
Congenital Heart Disease (CHD) is a leading cause of fetal mortality, making early detection via prenatal ultrasound a critical healthcare priority. However, developing robust deep learning models for CHD detection is hampered by two major challenges: extreme data imbalance and strict privacy regulations. Because CHD is relatively rare, collecting a sufficiently diverse dataset of positive cases for supervised learning is often impractical. Furthermore, data governance laws frequently prevent hospitals from pooling sensitive patient data into centralized databases for training.
This paper introduces a novel framework that addresses these issues by reframing CHD detection as a zero-shot anomaly detection task. By training models exclusively on healthy data and merging them using a privacy-preserving technique, the authors demonstrate a scalable method for detecting heart defects across diverse clinical environments.
Methodology
Sparse Tube Ultrasound Distillation (STUD)
To handle the high computational demands of video processing, the authors propose Sparse Tube Ultrasound Distillation (STUD). Rather than attempting to classify specific defects, the framework treats CHD detection as a video anomaly detection problem.
Models are trained at individual hospitals using only videos of healthy fetal hearts. The STUD approach utilizes a "sparse tube" sampling method, extracting 3D space-time tubes of varying sizes—some capturing long-duration motion and others focusing on fine spatial details. This is significantly more efficient than dense tokenization. These tokens are processed by a Vision Transformer using a self-distillation loss (similar to the DINO architecture) to learn the complex representations of a "normal" fetal heart.
Divergence Vector-guided Model Merging (DiVMerge)
To combine the insights from multiple hospitals without sharing raw data, the authors developed Divergence Vector-guided Model Merging (DiVMerge). This technique creates a robust global model by merging the weights of local models through three primary steps:
- Geometric Median: The framework calculates the geometric median of all local model weights. This acts as a robust "consensus" that filters out extreme outliers caused by site-specific noise.
- Divergence Vectors: The difference between each local model and the geometric median is calculated. The magnitude of this vector determines the model's weight; models closer to the consensus are prioritized.
- Selective Parameter Retention: At the parameter level, the algorithm only retains weights that show stable, confident deviations from the median, ensuring the final model remains stable and generalized.
During inference, the model extracts features from a new ultrasound clip and uses a K-Nearest Neighbors (KNN) algorithm to determine if the features deviate significantly from the learned "healthy" distribution. If they do, the case is flagged as a zero-shot anomaly (CHD).
Experimental Setup and Datasets
The study utilized real-world fetal heart ultrasound video sweeps from five different hospitals in the UK and Australia.
- Training & Internal Testing: Data from Sites 1, 2, and 3 were used to train local models exclusively on healthy heart videos.
- External Zero-Shot Testing: Sites 4 and 5 were reserved for external validation to test the model's ability to handle domain shift (different machines and operators). These test sets included various unseen CHD anomalies, such as Right Aortic Arch (RAA) and Ventricular Septal Defects (VSD).
Results and Discussion
Efficiency and Performance
The STUD tokenizer proved remarkably efficient, using only 9.39% of the tokens compared to standard video baselines like TimeFormer, while still achieving a higher average F1-score (82.20 vs 79.87) on internal sites.
Superiority Over Centralization
One of the most significant findings was that the merged model (DiVMerge) outperformed a model trained on a centralized pool of all data. Centralized models often struggle with "domain shift" between different hospitals. On external test sites (Sites 4 and 5), the centralized model failed, scoring only a 23.5% F1-score. In contrast, the DiVMerge model achieved an 81.0% accuracy and a 78.1% F1-score—a 54.6% improvement over the centralized baseline.
Comparison with Other Merging Techniques
DiVMerge also outperformed five other state-of-the-art model merging techniques, including Model Soup and Task Vectors. Notably, DiVMerge does not require a pre-trained base model to function, making it more flexible for specialized medical tasks.
Conclusion
The proposed framework offers a privacy-compliant, scalable alternative to centralized deep learning for rare disease detection. By focusing on normality modeling and intelligent weight merging, the authors have created a system that generalizes exceptionally well to unseen clinical environments. This approach holds great potential for smaller clinics, allowing them to leverage the diagnostic power of larger institutions without the risks associated with data sharing.