SiamQuality: A ConvNet-Based Foundation Model for Imperfect Physiological Signals

Cheng Ding, Zhicheng Guo, Zhaoliang Chen, Randall J. Lee, Cynthia Rudin, Xiao HuarXiv 2024 · DOI · arXiv

Abstract

Physiological signals from wearables and clinical monitors are often corrupted by noise and artifacts, posing a challenge for standard foundation models. We present SiamQuality, a CNN-based foundation model trained on 36 million signal pairs using a novel quality-pairing contrastive learning strategy. By leveraging temporal proximity to pair clean and noisy segments from the same patient, SiamQuality learns robust, quality-invariant representations. Our model achieves state-of-the-art performance in respiration rate estimation and atrial fibrillation detection, demonstrating significant resilience to data imperfections.

Introduction

Foundation models (FMs) have transformed natural language processing and computer vision by training on massive datasets. However, applying FMs to physiological signals—such as photoplethysmography (PPG) from wearables or ICU monitors—is difficult because real-world data is rarely "clean." Motion artifacts, sensor displacement, and environmental noise frequently degrade signal quality. Existing biosignal FMs often rely on complex Transformer architectures that struggle with these imperfections. SiamQuality addresses this gap by introducing a foundation model inherently robust to poor data quality, bridging the divide between ideal training conditions and clinical reality.

Methods

SiamQuality utilizes a Convolutional Neural Network (CNN) backbone, specifically ResNet, rather than the more common Transformer architecture. The core of the approach is the SimQuality self-supervised task, which employs a "quality-pairing" strategy.

Overview of the SiamQuality pairing strategy, where good and bad quality segments from the same patient form a positive pair for contrastive learning.

Quality-Pairing and SimSiam Architecture

Instead of standard data augmentations, the model leverages the temporal consistency of medical data. It identifies a "good quality" anchor signal and pairs it with a "bad quality" segment from the same patient within a 5-minute window. Since the underlying physiological state (e.g., heart rhythm) is unlikely to change drastically in such a short period, the model is forced to produce similar representations for both clean and noisy versions of the same state. This process teaches the model to ignore noise and focus on biological features.

The Siamese CNN architecture used in SiamQuality, featuring shared weights between encoders and a stop-gradient operation to prevent collapse.

The training uses a Siamese network structure that minimizes the distance between these pairs without requiring negative samples. Furthermore, a curriculum learning approach is applied, where the model is progressively exposed to pairs with increasing levels of noise.

Datasets and Evaluation

The model was pre-trained on a massive dataset from the UCSF Medical Center, comprising over 36 million 30-second PPG pairs (approx. 600,000 hours) from 21,000 patients. Downstream evaluation was conducted on several public benchmarks:

  • Heart Rate & Blood Pressure: TROIKA, Dalia, WESAD, and PulseDB.
  • Respiration Rate (RR): BIDMC PPG and Respiration Dataset.
  • Atrial Fibrillation (AF) Detection: Stanford AF dataset.

Results

SiamQuality established new state-of-the-art benchmarks across six downstream tasks. Notably, it achieved a Mean Absolute Error (MAE) of 0.89 for Respiration Rate, a 43% improvement over previous methods. In AF detection, the model reached an F1 score of 0.71 with 100% artifact coverage.

A key contribution is the introduction of the Artifact Tolerance Curve (AT-Curve), which visualizes performance degradation as signal quality decreases. SiamQuality demonstrated remarkable resilience; on the noisy DALIA dataset, it maintained high accuracy even when signals contained over 60% artifacts, whereas competing models failed. Additionally, the model exhibited clear scaling laws, with performance improving consistently as the ResNet depth increased from 50 to 152 layers.

Discussion and Limitations

While SiamQuality represents a significant step forward, several areas for future work remain. The model currently requires fine-tuning for specific tasks and does not support zero-shot execution. It also faces challenges with class imbalance in rare events like Premature Ventricular Contractions. Future iterations could benefit from more generalized quality assessment tools applicable to other modalities like ECG or EEG. Finally, clinical integration will require further engineering and validation through clinical trials.

Summary updated

Abstract

Foundation models, especially those using transformers as backbones, have gained significant popularity, particularly in language and language-vision tasks. However, large foundation models are typically trained on high-quality data, which poses a significant challenge, given the prevalence of poor-quality real-world data. This challenge is more pronounced for developing foundation models for physiological data; such data are often noisy, incomplete, or inconsistent. The present work aims to provide a toolset for developing foundation models on physiological data. We leverage a large dataset of photoplethysmography (PPG) signals from hospitalized intensive care patients. For this data, we propose SimQuality, a novel self-supervised learning task based on convolutional neural networks (CNNs) as the backbone to enforce representations to be similar for good and poor quality signals that are from similar physiological states. We pre-trained the SimQuality on over 36 million 30-second PPG pairs and then fine-tuned and tested on six downstream tasks using external datasets. The results demonstrate the superiority of the proposed approach on all the downstream tasks, which are extremely important for heart monitoring on wearable devices. Our method indicates that CNNs can be an effective backbone for foundation models that are robust to training data quality.
BibTeX
@misc{ding2024siamquality,
  author = {Cheng Ding and Zhicheng Guo and Zhaoliang Chen and Randall J. Lee and Cynthia Rudin and Xiao Hu},
  title = {{SiamQuality: A ConvNet-Based Foundation Model for Imperfect Physiological Signals}},
  year = {2024},
  eprint = {2404.17667},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2404.17667},
}
Figure