Aprimoramento do diagnóstico automatizado de eletrocardiograma (ECG) por meio de pré-treinamento multimodal com laudos em texto

Jose Geraldo Fernandes, Diogo Tuler, Gabriel Lemos, Pedro Robles Dutenhefner, Tércia Maria Ribeiro Lima Rezende, Gisele L. Pappa, Gabriela Paixão, Antônio H. Ribeiro, Wagner MeiraJournal of Health Informatics 2024 · DOI

Abstract

Heart diseases are the leading cause of death worldwide, and the electrocardiogram (ECG) is the primary diagnostic tool for assessing cardiac activity. Automated and remote ECG diagnosis can help the healthcare system with timely and high-quality cardiac assessments, especially for peripheral regions and rural areas. This study enhances the performance of ECG deep learning classification models using a multimodal pre-training stage with physician's reports. Our approach improves the state-of-the-art model and achieves a mean F1 score of 0.755 over six categories using the full dataset, which is a relevant improvement for a relatively larger unlabeled corpus. The results demonstrate the potential to improve automated cardiac assessment with text pretraining.

Introduction

Electrocardiograms (ECGs) are the gold standard for diagnosing heart disease, yet interpreting them accurately at scale remains a significant challenge. While deep learning has shown great promise in automated ECG classification, these models typically require massive amounts of expertly labeled data to perform well. In real-world clinical environments, labels are expensive and time-consuming to produce. However, these environments generate a wealth of "unlabeled" data: raw ECG signals paired with free-text medical reports.

The study integrates various medical data points, including cardiac signals and clinical narratives, to improve diagnostic accuracy.

This research addresses the challenge of leveraging existing text reports to "teach" a model the underlying patterns of cardiac electrical activity without needing explicit, categorical labels for every training sample.

Methodology: Multimodal Contrastive Learning

The researchers adopted a multimodal contrastive learning framework, inspired by the CLIP (Contrastive Language-Image Pre-training) architecture. The process is divided into two distinct phases:

  1. Multimodal Pre-training: The system utilizes two specialized encoders—a 1D ResNet for the ECG signal and BioBERTpt for the text (a model specialized for Portuguese clinical narratives). During this phase, the model learns to project both the signal and the text into a shared mathematical space. It is trained to maximize the similarity between a specific ECG signal and its corresponding report while minimizing similarity with unrelated reports. The text encoder remains "frozen," acting as a knowledgeable supervisor for the signal encoder.
  2. Supervised Fine-tuning: The pre-trained signal encoder is subsequently transferred to a supervised task, where it is trained on a smaller set of labeled data to classify specific heart conditions.

By using text as a "free" supervisor, the model captures rich, nuanced descriptions that simple binary labels (e.g., "Normal" vs. "Abnormal") often miss.

Dataset and Experimental Setup

The study utilized the CODE-15% dataset, a subset of the Clinical Outcomes in Digital Electrocardiology (CODE) dataset from the Telehealth Network of Minas Gerais, Brazil. This dataset includes 345,779 ECGs from over 233,000 patients, featuring 12-lead signals paired with Portuguese medical reports.

The models were evaluated on six specific abnormalities: 1st-degree AV block (1aAVb), Right Bundle Branch Block (RBBB), Left Bundle Branch Block (LBBB), Sinus Bradycardia (SB), Atrial Fibrillation (AF), and Sinus Tachycardia (ST).

Key Results

The multimodal approach consistently outperformed the "signal-only" baseline across various data scales:

  • State-of-the-Art Performance: On the full dataset, the model achieved a mean F1 score of 0.755 across all six categories.
  • Data Efficiency: The most significant advantage was observed in "data-scarce" scenarios. When the fine-tuning set was reduced to just 0.7% of the original data, the multimodal model maintained an F1 score of 0.518, while the baseline model's performance plummeted to 0.209.
  • Clinical Relevance: The model showed significant gains in identifying critical conditions like Atrial Fibrillation (AF), which is essential for stroke prevention.

Conclusion and Future Work

This study demonstrates that pre-training deep learning models on a combination of ECG signals and their corresponding physician reports significantly boosts diagnostic accuracy, particularly when labeled data is limited. The use of domain-specific NLP (BioBERTpt) was essential for accurately capturing medical semantics.

Future work includes exploring automated report generation to leverage even larger unlabeled datasets and moving toward integrating these models into remote diagnosis systems to prioritize care in underserved areas.

This research was presented at the XX Brazilian Congress of Health Informatics (CBIS'24).

Summary updated

Abstract

Objective: Heart diseases are the leading cause of death worldwide, and the electrocardiogram (ECG) is the primary diagnostic tool for assessing cardiac activity. Automated and remote ECG diagnosis can help the healthcare system with timely and high-quality cardiac assessments, especially for peripheral regions and rural areas. Automatic ECG classification has been extensively researched, but it is still challenging to build accurate models for such a wide spectrum of scenarios. Method: This study enhances the performance of ECG deep learning classification models using a multimodal pre-training stage with physician's reports. Results: Our approach improves the state-of-the-art model and achieves a mean F1 score of 0.755 over six categories using the full dataset, which is a relevant improvement for a relatively larger unlabeled corpus. Conclusion: The results demonstrate the potential to improve automated cardiac assessment with text pretraining.
BibTeX
@article{fernandes2024aprimoramento,
  author = {Jose Geraldo Fernandes and Diogo Tuler and Gabriel Lemos and Pedro Robles Dutenhefner and Tércia Maria Ribeiro Lima Rezende and Gisele L. Pappa and Gabriela Paixão and Antônio H. Ribeiro and Wagner Meira},
  title = {{Aprimoramento do diagnóstico automatizado de eletrocardiograma (ECG) por meio de pré-treinamento multimodal com laudos em texto}},
  journal = {Journal of Health Informatics},
  year = {2024},
  doi = {10.59681/2175-4411.v16.iespecial.2024.1368},
}
Figure