A Contrastive Predictive Coding-Based Classification Framework for Healthcare Sensor Data

Chaoxu Ren, Le Sun, Dandan PengJournal of Healthcare Engineering 2022 · DOI

TL;DR

This paper introduces TCC, a novel two-step framework for classifying healthcare sensor data using self-supervised learning. By leveraging Contrastive Predictive Coding (CPC) for a pretext task, TCC effectively learns robust features from unlabeled data, significantly improving classification accuracy, especially in low-data regimes, compared to traditional supervised methods.

What Problem Does This Paper Address?

The healthcare industry is increasingly relying on sensor data for automatic disease classification and real-time health monitoring. However, traditional supervised learning methods require vast amounts of manually annotated data, which is expensive and time-consuming to acquire, especially for medical data with high labeling thresholds. This bottleneck hinders the widespread adoption of AI in healthcare. The paper addresses this by proposing a self-supervised learning approach that can learn valuable representations from unlabeled sensor data, thereby reducing the reliance on extensive manual annotation and improving classification performance, particularly when labeled data is scarce.

Methods

The proposed framework, TCC (Two-step CPC-based Classification), operates in two stages:

  1. CPC-based Pretext Task: This stage employs Contrastive Predictive Coding (CPC) to learn generalizable features. Instead of predicting future data points, TCC innovates by designing a pretext task that focuses on predicting relationships between positive and negative sample pairs derived from the sensor data. Positive pairs consist of samples from the same category, while negative pairs are from different categories. The goal is to maximize the similarity between positive pairs and minimize it for negative pairs. This contrastive learning approach helps the encoder learn discriminative representations.
  2. Downstream Classification Task: In the second stage, the encoder trained in the pretext task is repurposed for a lightweight classification model. The encoder's learned parameters are loaded, and a simple classification architecture (e.g., Conv1 layers) is trained on top of these features. This two-step process allows the model to leverage the rich features learned during the self-supervised pretext task for efficient and accurate classification on downstream tasks.

Datasets

The paper evaluates the TCC framework on two types of medical sensor data:

  • Electroencephalogram (EEG) data: Specifically, the sleep-EDF dataset was used for sleep stage classification. This dataset contains 197 whole-night polysomnographic recordings, and the study utilized a single EEG channel (Fpz-Cz) sampled at 100 Hz. The sleep stages classified were Awake (W), N1, N2, N3, and Rapid Eye Movement (REM).
  • Electrocardiogram (ECG) data: The MIT-BIH supraventricular arrhythmia database (MIT-BIH-SUP) was used to assess the model's robustness and sustainability. This dataset includes ECG recordings to classify heartbeats into five classes: normal beat (N), supraventricular ectopic beat (S), ventricular ectopic beat (V), fusion beat (F), and unclassifiable beat (Q).

Key Insights and Ideas

  • Contrastive Learning for Sensor Data: Adapting CPC's contrastive learning principle to predict relationships between sample pairs (rather than just future states) proves effective for extracting meaningful features from time-series sensor data.
  • Pretext Task Quality is Crucial: The performance of the downstream classification model is highly dependent on the quality of features learned during the pretext task. Thorough training of the pretext task is emphasized.
  • Efficiency in Low-Data Regimes: The TCC framework demonstrates remarkable performance even with a small proportion of labeled data, significantly outperforming traditional supervised learning in such scenarios. This highlights its potential for real-world applications where data annotation is a bottleneck.
  • Transferability of Learned Features: The encoder trained on one dataset (EEG) can be effectively used for classification on another dataset (ECG), showcasing the generalizability of the learned representations.

Key Results

  • Sleep Stage Classification: The TCC framework achieved an overall accuracy of 88.70% and a macro-averaged F1-score of 88.09% on the sleep-EDF dataset. The F1-scores for individual sleep stages were also high, with the N1 category showing a particularly good classification effect.
  • Performance with Limited Labels: When tested with varying percentages of labeled data (10%, 30%, 50%, 70%, 100%), the TCC framework consistently maintained high accuracy, often exceeding supervised learning performance, especially at lower data percentages. For instance, at 10% labeled data, TCC achieved significantly higher accuracy than supervised learning.
  • Arrhythmia Classification: On the MIT-BIH-SUP dataset, TCC achieved an accuracy of 97.30% for classifying different heartbeat types, demonstrating its robustness and sustainability across different sensor data modalities.
  • Comparison to State-of-the-Art: The paper compares TCC to other models, showing competitive or superior performance in terms of accuracy (ACC) and macro-averaged F1-score (MF1). For example, TCC achieved an ACC of 88.70% and MF1 of 88.09%, outperforming models like DeepSleepNet and SleepEEGNet.

Limitations and Future Work

  • Model Complexity: While the downstream classification model is lightweight, the authors suggest exploring more complex or time-based classification models in the future to further enhance accuracy.
  • Sample Imbalance: Although sample imbalance did not significantly affect the final experimental results, the authors plan to investigate machine learning techniques like synthetic minority oversampling (SMOTE) to address this issue in future work.
  • Generalizability: While tested on two distinct datasets, further validation on a wider range of healthcare sensor data would be beneficial to fully establish the framework's generalizability.

Summary updated

Abstract

Supervised learning technologies have been used in medical-data classification to improve diagnosis efficiency and reduce human diagnosis errors. A large amount of manually annotated data are required for the fully supervised learning process. However, annotating data information will consume a large amount of manpower and resources. Self-supervised learning has great advantages in solving this problem. Self-supervised learning mainly uses pretext tasks to mine its own supervised information from large-scale unsupervised data. And this constructed supervised information is used to train the network to learn valuable representations for downstream tasks. This study designs a general and efficient model for the diagnosis and classification of medical sensor data based on contrastive predictive coding (CPC) in self-supervised learning, called TCC, which consists of two steps. The first step is to design a pretext task based on the idea of CPC, which aims to extract effective features between different categories using its encoder. The second step designs a downstream classification task with lower time and space complexity to perform a supervised type of training using the features extracted by the encoder of the pretext task. Finally, to demonstrate the performance of the proposed framework in this paper, we compare the proposed framework with recent state-of-the-art works. Experiments comparing the proposed framework with supervised learning are also set up under the condition of different proportions of labeled data.
BibTeX
@article{ren2022contrastive,
  author = {Chaoxu Ren and Le Sun and Dandan Peng},
  title = {{A Contrastive Predictive Coding-Based Classification Framework for Healthcare Sensor Data}},
  journal = {Journal of Healthcare Engineering},
  year = {2022},
  doi = {10.1155/2022/5649253},
}
Figure