Abstract
Heart diseases are the leading cause of death worldwide, and the electrocardiogram (ECG) is the primary diagnostic tool for assessing cardiac activity. Automated and remote ECG diagnosis can help the healthcare system with timely and high-quality cardiac assessments, especially for peripheral regions and rural areas. This study enhances the performance of ECG deep learning classification models using a multimodal pre-training stage with physician's reports. Our approach improves the state-of-the-art model and achieves a mean F1 score of 0.755 over six categories using the full dataset, which is a relevant improvement for a relatively larger unlabeled corpus. The results demonstrate the potential to improve automated cardiac assessment with text pretraining.
Introduction
Electrocardiograms (ECGs) are the gold standard for diagnosing heart disease, yet interpreting them accurately at scale remains a significant challenge. While deep learning has shown great promise in automated ECG classification, these models typically require massive amounts of expertly labeled data to perform well. In real-world clinical environments, labels are expensive and time-consuming to produce. However, these environments generate a wealth of "unlabeled" data: raw ECG signals paired with free-text medical reports.
This research addresses the challenge of leveraging existing text reports to "teach" a model the underlying patterns of cardiac electrical activity without needing explicit, categorical labels for every training sample.
Methodology: Multimodal Contrastive Learning
The researchers adopted a multimodal contrastive learning framework, inspired by the CLIP (Contrastive Language-Image Pre-training) architecture. The process is divided into two distinct phases:
- Multimodal Pre-training: The system utilizes two specialized encoders—a 1D ResNet for the ECG signal and BioBERTpt for the text (a model specialized for Portuguese clinical narratives). During this phase, the model learns to project both the signal and the text into a shared mathematical space. It is trained to maximize the similarity between a specific ECG signal and its corresponding report while minimizing similarity with unrelated reports. The text encoder remains "frozen," acting as a knowledgeable supervisor for the signal encoder.
- Supervised Fine-tuning: The pre-trained signal encoder is subsequently transferred to a supervised task, where it is trained on a smaller set of labeled data to classify specific heart conditions.
By using text as a "free" supervisor, the model captures rich, nuanced descriptions that simple binary labels (e.g., "Normal" vs. "Abnormal") often miss.
Dataset and Experimental Setup
The study utilized the CODE-15% dataset, a subset of the Clinical Outcomes in Digital Electrocardiology (CODE) dataset from the Telehealth Network of Minas Gerais, Brazil. This dataset includes 345,779 ECGs from over 233,000 patients, featuring 12-lead signals paired with Portuguese medical reports.
The models were evaluated on six specific abnormalities: 1st-degree AV block (1aAVb), Right Bundle Branch Block (RBBB), Left Bundle Branch Block (LBBB), Sinus Bradycardia (SB), Atrial Fibrillation (AF), and Sinus Tachycardia (ST).
Key Results
The multimodal approach consistently outperformed the "signal-only" baseline across various data scales:
- State-of-the-Art Performance: On the full dataset, the model achieved a mean F1 score of 0.755 across all six categories.
- Data Efficiency: The most significant advantage was observed in "data-scarce" scenarios. When the fine-tuning set was reduced to just 0.7% of the original data, the multimodal model maintained an F1 score of 0.518, while the baseline model's performance plummeted to 0.209.
- Clinical Relevance: The model showed significant gains in identifying critical conditions like Atrial Fibrillation (AF), which is essential for stroke prevention.
Conclusion and Future Work
This study demonstrates that pre-training deep learning models on a combination of ECG signals and their corresponding physician reports significantly boosts diagnostic accuracy, particularly when labeled data is limited. The use of domain-specific NLP (BioBERTpt) was essential for accurately capturing medical semantics.
Future work includes exploring automated report generation to leverage even larger unlabeled datasets and moving toward integrating these models into remote diagnosis systems to prioritize care in underserved areas.
Summary updated