Self-supervised learning-based general laboratory progress pretrained model for cardiovascular event detection

Li-Chin Chen, Kuo-Hsuan Hung, Yi-Ju Tseng, Hsin-Yao Wang, Tse-Min Lu, Wei-Chieh Huang, Yu Tsao

TL;DR

This paper introduces a Generalized Laboratory Progress (GLP) model pretrained via self-supervised learning on large-scale, irregular longitudinal health data to capture disease progression trends. By transferring this learned representation to a smaller, episodic dataset, the model drastically improves the detection accuracy of adverse cardiovascular events (Target Vessel Revascularization) from 0.63 to 0.90.

What Problem Does This Paper Address?

Electronic Health Records (EHRs) contain a wealth of longitudinal laboratory data, but it is notoriously messy—characterized by irregular testing intervals, missing values, and high sparsity. While large patient cohorts (e.g., those with chronic hypertension) accumulate enough data to potentially train machine learning models despite these flaws, rare diseases or specific post-operative events suffer from small sample sizes and purely episodic, cross-sectional observations.

For instance, predicting Target Vessel Revascularization (TVR)—the need to re-clear a stented artery after a percutaneous coronary intervention (PCI)—is critical for patient prognosis. However, data at the time of PCI is typically a single "snapshot," making it incredibly difficult to train robust predictive models for these high-risk cardiac events. The authors address this data scarcity gap by transferring disease progression knowledge from a data-rich, prevalent patient population to a data-sparse, specific one.

Methods

The authors propose a two-stage self-supervised pretraining framework called the Generalized Laboratory Progress (GLP) model.

  1. Data Interpolation & Framing: To handle missing data in the longitudinal timeline, they evaluate three interpolation techniques (linear, PCHIP, and barycentric) and organize the patient data into 12-month overlapping windows (frames).
  2. Stage 1 (Interpolation Training): The model learns general laboratory progress trends using the interpolated data in a supervised manner.
  3. Stage 2 (Self-Supervised Refinement): The model uses an autoregressive self-supervised learning (SSL) approach on non-interpolated data, predicting future lab values based on its own previous generations to refine its temporal latent representations.
Longitudinal data framing for the two-stage training process

The core architecture consists of a Longitudinal Iterative Block (LIBC)—combining a Bidirectional LSTM with a fully connected condensing layer—and a regressor. Six separate GLP models are trained individually for six key lab markers: Chol/HDL-c ratio, LDL-c, LDL-c/HDL-c ratio, fasting blood glucose (Glucose AC), white blood cell count (WBC), and uric acid (UA).

Fig. 4. Training pipeline. GLP: general laboratory progress pretrain model; Chol/HDL-c: ratio of total cholesterol and high-density lipoprotein choles- terol; LDL-c: low-density lipoprotein cholesterol; LDL-c/HDL-c: ratio of low- density and high-density lipoprotein cholesterol; Glucose AC: fasting blood sugar; WBC: white blood cells; UA: Uric Acid; PCI: percutaneous coronary intervention; TVR: target vessel revascularization.

Once pretrained, these six models are frozen and their outputs are concatenated into a multimodal representation. Simple, non-neural network classifiers (like LightGBM, SVM, and Logistic Regression) are then trained on these distilled features to predict downstream TVR events.

Datasets

The study utilizes two distinct datasets from different medical institutions to ensure the generality of the findings:

  • Pretext Dataset (Pretraining): Sourced from the Chang Gung Research Database (CGRD) in Taiwan, containing longitudinal records of 9,720 patients who progressed from hypertension (HTN) to diabetes mellitus (DM). This provided the large-scale, continuous data needed to model general lab marker trajectories.
  • Downstream Dataset (Evaluation): Sourced from Taipei Veterans General Hospital, containing episodic records of 483 patients who underwent PCI. This dataset tracks the occurrence of TVR. Due to severe class imbalance (42 TVR vs. 441 non-TVR), the negative cases were randomly downsampled to create a balanced evaluation set of 84 patients.

Key Insights and Ideas

  • Translational Knowledge Transfer: The core "aha" moment is that disease progression dynamics can be successfully transferred across completely different patient cohorts. Trends learned from a general hypertension-to-diabetes cohort can help predict acute stent failures in a completely separate post-PCI cohort because of shared underlying cardiometabolic risk pathways.
  • The Distillation Effect: The authors compared two latent outputs: the internal embedding of the LSTM ($Progress_{emb}$) and the final output of the regressor ($Progress_{out}$). $Progress_{out}$ performed significantly better because it acts as a distilled trend indicator focused purely on future trajectories, stripping away confounding static variables.
  • Geometrical Transformation of Data: GLP completely alters the geometry of the feature space. Before processing, TVR and non-TVR cases are hopelessly entangled. After GLP processing, stable (non-TVR) cases tightly converge to a singular point, while high-risk (TVR) cases remain scattered, making classification trivial for tree-based models like LightGBM.

Key Results

Integrating the GLP model led to a massive performance leap in downstream TVR detection:

  • Accuracy Boost: The average classification accuracy across all tested machine learning models soared from 0.63 (using raw, un-preprocessed data) to 0.90 when using the distilled $Progress_{out}$ representation.
  • High Discriminative Power: The average AUROC reached 0.91, with LightGBM achieving an individual AUROC of 0.973 and an F1-score of 0.952.
  • Interpolation Matters: Linear and PCHIP interpolation methods significantly outperformed barycentric interpolation during the pretraining phase, aligning with the idea that biological lab markers progress along continuous curves rather than discrete clusters.

Limitations and Future Work

  • Numeric Only: The current GLP model exclusively handles numeric laboratory results and cannot incorporate categorical clinical variables (e.g., smoking status, specific comorbidities).
  • Lack of Invertibility: The output is a latent representation; it cannot currently be mapped back into real-world clinical values (e.g., predicting an exact blood sugar mg/dL value) without adding a dedicated decoder network.
  • Interpretability Trade-off: Like many deep learning and SSL approaches, the model sacrifices clinical interpretability for high predictive accuracy, which can pose a barrier to adoption in real-world clinical workflows where physicians demand to know why a patient is flagged as high-risk.

Summary updated

Abstract

The inherent nature of patient data poses several challenges. Prevalent cases amass substantial longitudinal data owing to their patient volume and consistent follow-ups, however, longitudinal laboratory data are renowned for their irregularity, temporality, absenteeism, and sparsity; In contrast, recruitment for rare or specific cases is often constrained due to their limited patient size and episodic observations. This study employed self-supervised learning (SSL) to pretrain a generalized laboratory progress (GLP) model that captures the overall progression of six common laboratory markers in prevalent cardiovascular cases, with the intention of transferring this knowledge to aid in the detection of specific cardiovascular event. GLP implemented a two-stage training approach, leveraging the information embedded within interpolated data and amplify the performance of SSL. After GLP pretraining, it is transferred for TVR detection. The proposed two-stage training improved the performance of pure SSL, and the transferability of GLP exhibited distinctiveness. After GLP processing, the classification exhibited a notable enhancement, with averaged accuracy rising from 0.63 to 0.90. All evaluated metrics demonstrated substantial superiority (p < 0.01) compared to prior GLP processing. Our study effectively engages in translational engineering by transferring patient progression of cardiovascular laboratory parameters from one patient group to another, transcending the limitations of data availability. The transferability of disease progression optimized the strategies of examinations and treatments, and improves patient prognosis while using commonly available laboratory parameters. The potential for expanding this approach to encompass other diseases holds great promise.
BibTeX
@article{chen2023self,
  author = {Li-Chin Chen and Kuo-Hsuan Hung and Yi-Ju Tseng and Hsin-Yao Wang and Tse-Min Lu and Wei-Chieh Huang and Yu Tsao},
  title = {{Self-supervised learning-based general laboratory progress pretrained model for cardiovascular event detection}},
  journal = {IEEE Journal of Translational Engineering in Health and Medicine},
  year = {2023},
  doi = {10.1109/jtehm.2023.3307794},
}
Figure