Weakly Supervised Classification of Vital Sign Alerts as Real or Artifact

Arnab Dey, Mononito Goswami, Joo Heung Yoon, Gilles Clermont, Michael R. Pinsky, Marilyn Hravnak, Artur Dubrawski

Introduction

In modern Intensive Care Units (ICUs), clinicians are frequently overwhelmed by physiological alarms, often receiving up to 1,000 alerts per shift. A significant majority of these are false alarms—artifacts caused by patient movement, electrical interference, or sensor displacement. This phenomenon, known as "alarm fatigue," desensitizes medical staff and poses a critical risk to patient safety, as life-threatening events may be overlooked.

While machine learning offers a potential solution for filtering these artifacts, traditional supervised learning requires vast amounts of expert-labeled "ground truth" data. Manually reviewing waveform data is prohibitively expensive and slow. This study explores a Weak Supervision (WS) framework to bypass this bottleneck by encoding clinical heuristics into programmatic labeling functions.

Methodology

The researchers implemented a Weak Supervision pipeline using the Snorkel framework. This approach shifts the focus from manual labeling to the creation of "Labeling Functions" (LFs) that automate the labeling process.

  1. Heuristic Harvesting: Clinicians define "business rules" based on physiological intuition (e.g., cross-referencing heart rate from ECG with pulse oximetry).
  2. Labeling Functions: These rules are converted into Python functions that assign labels (Real, Artifact, or Abstain) to the data.
  3. Generative Label Model: A model estimates the accuracy and correlations of these noisy LFs to produce probabilistic labels.
  4. Downstream Classifier: A Random Forest (RF) model is trained on these probabilistic labels, utilizing features extracted from high-density waveform data.
The weak supervision pipeline: from clinician heuristics to a trained Random Forest classifier.

Data and Features

The study utilized a large single-center database comprising 367,464 monitoring hours. The analysis focused on two primary alert types:

  • Respiratory Rate (RR) Alerts: 648 one-minute windows.
  • Oxygen Saturation ($SpO_2$) Alerts: 621 one-minute windows.

The models utilized both raw numeric data and features derived from high-density waveforms, such as pulsatility, spectral analysis (FFT), and peak detection. Access to raw waveform data proved critical, particularly for $SpO_2$ alerts, as the signal's morphology often reveals artifacts that summary numerics obscure.

Results

The weakly supervised models performed competitively with, and in some instances outperformed, fully supervised models trained on manual labels.

  • Respiratory Rate (RR): The Weak Sup. model achieved an accuracy of 0.915 and an AUC of 0.951, notably surpassing the fully supervised model (Accuracy: 0.886, AUC: 0.898).
  • Oxygen Saturation ($SpO_2$): The Weak Sup. model achieved an accuracy of 0.881 and an AUC of 0.940, nearly matching the fully supervised benchmark (Accuracy: 0.903, AUC: 0.964).
Log-scale ROC plots showing the performance of the Weak Supervision pipeline across different alert types and conditions.

As shown in the ROC curves, the WS pipeline maintains high True Positive Rates even at very low False Positive Rates, a requirement for clinical safety to ensure real emergencies are not filtered out.

Discussion and Conclusion

Weak Supervision serves as a "force multiplier" for healthcare machine learning, allowing for the rapid development of high-performing models without the need for exhaustive manual labeling. The study demonstrates that models trained on noisy, heuristic-based labels can capture the underlying physiological logic as effectively as those trained on expert ground truth.

Future work will focus on transitioning from retrospective analysis to real-time deployment at the bedside, addressing the latency requirements necessary for live clinical decision support. While the system is dependent on the quality of the initial expert heuristics, it represents a significant step toward mitigating alarm fatigue in the ICU.

Summary updated

BibTeX
@inproceedings{dey2022weakly,
  author = {Arnab Dey and Mononito Goswami and Joo Heung Yoon and Gilles Clermont and Michael R. Pinsky and Marilyn Hravnak and Artur Dubrawski},
  title = {{Weakly Supervised Classification of Vital Sign Alerts as Real or Artifact}},
  booktitle = {AMIA},
  year = {2022},
  url = {https://openreview.net/forum?id=YOy4URSt97Z},
}

Loading related content...

Figure