TL;DR
This paper introduces PLITA, a novel self-supervised learning (SSL) method for single-lead ECG signals that simultaneously learns both invariant (stable over time) and tempo-variant (changing over time) attributes. By capturing these distinct signal characteristics, PLITA significantly improves performance on downstream tasks like AFib and sleep stage classification, demonstrating the crucial role of tempo-variant information often overlooked by existing methods.
What Problem Does This Paper Address?
Wearable sensors are increasingly vital for digital health, with single-lead ECG signals offering a rich source of information for diagnosing cardiac conditions. However, effectively extracting this information from unlabelled ECG data is challenging. Existing self-supervised learning (SSL) methods for ECG primarily focus on learning "invariant" attributes – stable characteristics of a person's heart signal that might indicate gender or predisposition to certain conditions. While valuable, this approach neglects "tempo-variant" attributes, which capture the dynamic, time-evolving nature of the heart's function, including adaptations to physiological needs or the onset of arrhythmias. This omission limits the model's ability to detect transient events like Atrial Fibrillation (AFib) or classify sleep stages, where these dynamic changes are critical. This paper aims to bridge this gap by developing an SSL method that explicitly learns both invariant and tempo-variant attributes.
Methods
The core innovation of PLITA lies in its Parallel-Learning framework, built upon the BYOL (Bootstrap Your Own Latent) architecture. It employs two networks: a student and a teacher (though the teacher is omitted in illustrations for clarity). PLITA introduces two distinct loss functions:
- Invariant Loss ($L_{iv}$): Similar to existing methods like PCLR, this loss encourages representations of ECG segments from different recordings of the same subject to be similar. This captures the stable, invariant characteristics.
- Tempo-variant Loss ($L_{tv}$): This is the novel component. It enforces that representations of ECG segments that are temporally proximate within the same recording should be closer in the representation space than those that are temporally distant. This is achieved by comparing the learned representations to an "Ideal Tempo-variant Distance Matrix" ($M^{tv}$), which defines a desired spatial ordering based on temporal proximity.
By using separate projectors for invariant ($G_{iv}$) and tempo-variant ($G_{tv}$) attributes, PLITA aims to disentangle these two types of information, preventing conflicting learning objectives.
Datasets
The paper evaluates PLITA on several publicly available datasets:
- MIT-BIH Atrial Fibrillation Database (MIT-AFIB): Used for AFib classification and disentangling studies.
- MIT-BIH Polysomnographic Database (MIT-PSG): Used for sleep stage classification.
- Physionet Challenge 2017 (Cinc2017): Used for AFib classification.
- Sleep Heart Health Study (SHHS): Used for pre-training and evaluation.
- Icentia: Used for pre-training and evaluation.
The paper does not introduce new datasets but utilizes established benchmarks for evaluating its method.
Key Insights and Ideas
- Tempo-variant attributes are not just noise: Existing SSL methods often treat tempo-variant signals as natural variance to be ignored when learning invariant features. PLITA demonstrates that these dynamic attributes are crucial and contain distinct, valuable information.
- Disentangling is key: By using separate projectors for invariant and tempo-variant attributes, PLITA effectively learns both types of information without them interfering with each other.
- Temporal proximity matters: The novel $L_{tv}$ loss function successfully encodes the idea that signals closer in time should have closer representations, capturing the heart's dynamic behavior.
- Holistic representation is superior: For tasks involving transient events or dynamic states (like AFib or sleep stages), a representation that captures both stable and changing aspects of the ECG signal leads to significantly better performance.
Key Results
- AFib Classification: PLITA achieved the highest accuracy (80.7%) and F1 score (78.4%) on the SHHS dataset for AFib classification, outperforming state-of-the-art methods like PCLR, CLOCS, and Mix-Up.
- Sleep Stage Classification: PLITA also demonstrated superior performance in sleep stage classification, achieving higher accuracy and AUC scores compared to baselines.
- Gender Identification: While not its primary focus, PLITA showed competitive results in gender classification, indicating that it still effectively captures invariant attributes.
- Representation Study: Disentangling studies confirmed that PLITA successfully separates invariant and tempo-variant features, with these features being consistent across different recordings of the same subject. SHAP analysis further highlighted the importance of both invariant and tempo-variant features for different tasks.
Limitations and Future Work
- Architecture Specificity: PLITA was evaluated using only the Vision Transformer (ViT) architecture. The authors suggest that the $L_{tv}$ loss function is agnostic to the specific architecture and could be applied to other non-contrastive frameworks like DINO or VIC-REG.
- Invariant Feature Extractor: Only BYOL was used as the base for capturing invariant features. Exploring other non-contrastive methods for invariant feature extraction could be a future direction.
The authors believe that incorporating tempo-variant information as a training objective will inspire future SSL work not only for ECG analysis but also for other time-series data.
Summary updated