Abstract
Ubiquitous sensing from wearable devices in the wild holds promise for enhancing human well-being, from diagnosing clinical conditions and measuring stress to building adaptive health promoting scaffolds. But the large volumes of data therein across heterogeneous contexts pose challenges for conventional supervised learning approaches. Representation Learning from biological signals is an emerging realm catalyzed by the recent advances in computational modeling and the abundance of publicly shared databases. The electrocardiogram (ECG) is the primary researched modality in this context, with applications in health monitoring, stress and affect estimation. Yet, most studies are limited by small-scale controlled data collection and over-parameterized architecture choices. We introduce WildECG, a pre-trained state-space model for representation learning from ECG signals. We train this model in a self-supervised manner with 275,000 10s ECG recordings collected in the wild and evaluate it on a range of downstream tasks. The proposed model is a robust backbone for ECG analysis, providing competitive performance on most of the tasks considered, while demonstrating efficacy in low-resource regimes.
Introduction
The proliferation of wearable devices like smartwatches and rings has enabled continuous, "in-the-wild" monitoring of electrocardiogram (ECG) signals. Unlike clinical 12-lead ECGs, these single-lead recordings are often noisy and lack expert labels. Furthermore, the high-frequency nature of ECG data creates a "quadratic bottleneck" for traditional Transformer architectures, where computational costs scale poorly with sequence length.
To address these challenges, we present WildECG, a foundation model designed for efficient and robust representation learning from ubiquitous ECG data. WildECG leverages Structured State-Space (S4) layers to capture long-range dependencies in heart activity without the prohibitive costs of self-attention.
Methods
WildECG's architecture is centered around the Structured State-Space (S4) backbone. S4 models are uniquely suited for time-series data as they combine the parallelizable training of Convolutional Neural Networks (CNNs) with the efficient inference of Recurrent Neural Networks (RNNs).
The model consists of a linear encoder, followed by six S4 blocks with residual connections, and an average pooling layer leading to a linear decoder. To train the model without manual labels, we employed a self-supervised "distortion detection" proxy task. Raw ECG signals from the TILES dataset were subjected to various transformations (e.g., noise addition, time warping, masking), and the model was tasked with identifying the specific transformation applied. This process encourages the model to learn intrinsic physiological features of the ECG signal.
Datasets and Evaluation
The model was pre-trained on the TILES dataset, which contains approximately 275,000 10-second recordings from 168 individuals in high-stress workplace environments.
We evaluated WildECG across several downstream tasks:
- Affect/Emotion: AVEC-16 and CASE datasets.
- Stress/Activity: WESAD and SWELL-KW datasets.
- Clinical Diagnosis: PTB-XL (large-scale diagnosis) and LUDB (heartbeat wave delineation).
Results
WildECG demonstrates significant advantages in efficiency and performance:
- Parameter Efficiency: With only ~313k parameters, WildECG is significantly smaller than standard ResNets or Transformers, making it ideal for edge deployment on wearables.
- Superior Performance: On the WESAD dataset, WildECG achieved 96.7% accuracy for stress detection, outperforming Transformers and specialized ECG networks.
- Clinical Diagnosis: On the PTB-XL dataset, the model improved upon the 1-lead S4 baseline by 1.3 percentage points in AUROC, showing that "wild" pre-training aids clinical tasks.
- Low-Resource Robustness: The model maintains high performance even when only 5% of downstream training data is available, highlighting the strength of its pre-trained representations.
Conclusion
WildECG provides a robust, efficient backbone for ECG analysis in real-world settings. While the model shows strong performance, future work will focus on integrating multimodal signals (e.g., EDA, physical activity) and mitigating subject-specific biases. By scaling representation learning to ubiquitous sensing, WildECG paves the way for more accessible and proactive health monitoring.
Summary updated