Abstract
Clinical 12-lead electrocardiography (ECG) is one of the most widely encountered kinds of biosignals. Despite the increased availability of public ECG datasets, label scarcity remains a central challenge in the field. Self-supervised learning represents a promising way to alleviate this issue. In this work, we put forward the first comprehensive assessment of self-supervised representation learning from clinical 12-lead ECG data. To this end, we adapt state-of-the-art self-supervised methods based on instance discrimination and latent forecasting to the ECG domain. In a first step, we learn contrastive representations and evaluate their quality based on linear evaluation performance on a recently established, comprehensive, clinical ECG classification task. In a second step, we analyze the impact of self-supervised pretraining on finetuned ECG classifiers as compared to purely supervised performance. For the best-performing method, an adaptation of contrastive predictive coding, we find a linear evaluation performance only 0.5% below supervised performance. For the finetuned models, we find improvements in downstream performance of roughly 1% compared to supervised performance, label efficiency, as well as robustness against physiological noise. This work clearly establishes the feasibility of extracting discriminative representations from ECG data via self-supervised learning and the numerous advantages when finetuning such representations on downstream tasks as compared to purely supervised training.
Introduction
Electrocardiograms (ECGs) are fundamental to cardiac diagnostics. While large volumes of unlabeled ECG data are increasingly available, obtaining expert clinical labels remains expensive and time-consuming. This "label scarcity" bottleneck hinders the development of robust deep learning models for ECG interpretation.
This study investigates whether self-supervised learning (SSL)—which has revolutionized computer vision and natural language processing—can be effectively ported to the 1D, multi-channel world of clinical ECGs. The goal is to create "foundation" representations that make models more accurate, more label-efficient, and more resilient to common clinical noise like muscle tremors or baseline wander.
Methodology
The researchers evaluated two primary paradigms of self-supervised learning adapted for ECG signals:
- Instance Discrimination (SimCLR, BYOL, SwAV): These methods learn representations by maximizing agreement between different augmented "views" of the same ECG record. The most effective augmentations were found to be "Time Out" (zeroing random segments) and "Random Resized Crop."
- Latent Forecasting (Contrastive Predictive Coding - CPC): CPC treats ECGs as temporal sequences. The model is trained to predict the latent representation of future segments based on past context. The authors enhanced the standard CPC architecture with a multi-layer perceptron (MLP) head and a specialized two-step finetuning protocol.
All models utilized the xresnet1d architecture, a 1D adaptation of ResNet, as the backbone encoder.
Results and Discussion
Comparative Performance
Unlike computer vision, where instance discrimination methods often dominate, CPC (latent forecasting) emerged as the superior method for ECG data. This suggests that the temporal, sequential nature of cardiac signals provides a more powerful supervisory signal for unsupervised learning than simple instance identity.
As shown in the performance analysis, CPC-based finetuning achieved a macro AUC of 0.9418, surpassing the best purely supervised baseline (0.9318) by 1% absolute. Even in "linear evaluation" (where the pretrained encoder is frozen), CPC reached a macro AUC of 0.927, just 0.5% shy of a fully supervised model.
Label Efficiency and Robustness
SSL significantly reduces the amount of labeled data required to achieve high performance. A CPC-pretrained model can match the performance of a standard supervised model using significantly less labeled training data.
Furthermore, SSL improves model robustness. By incorporating physiological noise transformations during pretraining, models became significantly more resistant to artifacts like muscle noise and baseline wander. The biggest performance gains were observed in "hard" classes—rare diseases and categories where supervised models typically struggle, highlighting the clinical utility of this approach.
Conclusion
This work establishes the feasibility and advantages of self-supervised representation learning for 12-lead ECG data. By leveraging unlabeled signals, we can build models that are more accurate, data-efficient, and robust. While CPC is computationally more intensive than instance discrimination methods, its superior performance on temporal biosignals makes it a compelling choice for future research in medical signal processing. Future work will explore scaling these methods to even larger datasets and more fine-grained clinical hierarchies.
Summary updated