DDL: Deep Dictionary Learning for Predictive Phenotyping

Tianfan Fu, Trong Nghia Hoang, Cao Xiao, Jimeng Sun

Introduction

Predictive phenotyping—the use of a patient's historical medical data to forecast future clinical events such as heart failure or mortality—is a critical component of modern healthcare analytics. While deep learning models have demonstrated significant potential in this domain, they are frequently hampered by the "label insufficiency" problem. Annotating Electronic Health Record (EHR) data requires expensive, expert medical knowledge, leaving vast amounts of clinical data unlabeled and underutilized.

Existing supervised models ignore this "dark matter" of unlabeled data, while traditional semi-supervised approaches often suffer from biased representations by training unsupervised and supervised components in isolation. This paper introduces Deep Dictionary Learning (DDL), a framework designed to bridge this gap by combining the temporal modeling capabilities of Recurrent Neural Networks (RNNs) with the parsimonious, interpretable representations of dictionary learning.

The DDL Framework

DDL integrates three interconnected modules into a unified architecture that leverages both labeled and unlabeled data through collaborative training.

  1. Deep Dictionary Encoder: Variable-length EHR sequences are first processed by a Long Short-Term Memory (LSTM) network to generate a fixed-size embedding, $h(x)$. This embedding is then decomposed into a set of "patient prototypes" (the dictionary $D$) and a patient-specific sparse code ($r$).
  2. Deep Dictionary Decoder: This module reconstructs the original clinical record from the sparse code. By forcing the model to reconstruct the input, the dictionary is compelled to capture the most fundamental and essential patterns within the EHR data.
  3. Deep Dictionary Predictor: A fully connected layer maps the learned sparse codes to target clinical outcomes, such as diagnosis or mortality risk.
The DDL architecture integrates an LSTM-based encoder with dictionary learning for joint reconstruction and prediction.

Collaborative Training

The core innovation of DDL is its joint optimization strategy. Unlike sequential methods, DDL optimizes a combined loss function encompassing projection, reconstruction, and prediction losses. This ensures that the latent representations (the dictionary and sparse codes) are simultaneously descriptive of the data and discriminative for the clinical task. To handle the non-differentiable $L_1$ regularization required for sparse coding, the authors utilize proximal gradient descent.

Experimental Evaluation

The model was rigorously evaluated on three large-scale healthcare datasets: Heart Failure (HF) prediction (16,794 samples), MIMIC-III mortality prediction (58,000 samples), and Truven MarketScan sequential prediction (72,179 samples).

Key Results and Insights

DDL consistently outperformed state-of-the-art baselines, including RETAIN and standard RNNs, across all tasks.

  • Performance under Scarcity: DDL's primary advantage is observed when labeled data is extremely limited. As shown in the performance analysis on the Heart Failure dataset, DDL maintains a significant lead over supervised baselines when only 1% to 10% of labels are available.
  • Unlabeled Data as a Regularizer: Experiments demonstrated that increasing the volume of unlabeled data consistently improves DDL's performance, whereas purely supervised models remain stagnant. This suggests that the reconstruction task acts as a powerful regularizer, preventing overfitting on small labeled sets.
  • Joint Optimization Benefits: Ablation studies confirmed that the inclusion of the decoder module is crucial. Without the reconstruction task to "anchor" the latent space, the model's performance drops, highlighting the importance of learning representations that are grounded in the actual data distribution.
Performance comparison (Average AUC) on the Heart Failure dataset across varying percentages of labeled training data.

Discussion and Limitations

While DDL offers superior accuracy and robustness in low-label regimes, it introduces certain trade-offs. The iterative proximal optimization required for sparse coding increases training time by approximately 2-3 times compared to standard RNNs. Additionally, the model requires careful tuning of trade-off coefficients to balance the different loss components.

Future work could focus on improving the clinical interpretability of the learned "prototypes" and optimizing the computational efficiency of the sparse coding step to facilitate real-time clinical deployment.

Conclusion

DDL represents a significant step forward in semi-supervised predictive phenotyping. By unifying deep temporal modeling with dictionary learning, it effectively transforms unlabeled EHR data into a valuable resource for improving clinical predictions, offering a robust solution for healthcare environments where expert labels are scarce.

Summary updated

Abstract

Predictive phenotyping is about accurately predicting what phenotypes will occur in the next clinical visit based on longitudinal Electronic Health Record (EHR) data. Several deep learning (DL) models have demonstrated great performance in predictive phenotyping. However, these DL-based phenotyping models requires access to a large amount of labeled data, which are often expensive to acquire. To address this label-insufficient challenge, we propose a deep dictionary learning framework (DDL) for phenotyping, which utilizes unlabeled data as a complementary source of information to generate a better, more succinct data representation. With extensive experiments on multiple real-world EHR datasets, we demonstrated DDL can outperform the state of the art predictive phenotyping methods on a wide variety of clinical tasks that require patient phenotyping such as heart failure classification, mortality prediction, and sequential prediction. All empirical results consistently show that unlabeled data can indeed be used to generate better data representation, which helps improve DDL's phenotyping performance over existing baseline methods that only uses labeled data.
BibTeX
@inproceedings{fu2019ddl,
  author = {Tianfan Fu and Trong Nghia Hoang and Cao Xiao and Jimeng Sun},
  title = {{DDL: Deep Dictionary Learning for Predictive Phenotyping}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2019},
  doi = {10.24963/ijcai.2019/812},
}
Figure