Abstract
Predicting patient mortality in Intensive Care Units (ICUs) is a critical task for clinical decision support. This paper introduces an end-to-end neural architecture that jointly learns to predict mortality and extract latent topics from clinical notes. By integrating Long Short-Term Memory (LSTM) networks with LDA-inspired topic modeling layers, the proposed models significantly outperform traditional two-stage baselines. This joint training approach ensures that the discovered topics are optimized for the predictive task, offering a path toward more interpretable clinical models.
Introduction
Mortality prediction in the ICU is essential for prioritizing care and managing resources. However, clinical data presents two major hurdles: temporal complexity and the need for accountability. Patient health is a dynamic trajectory, requiring models that can process sequential clinical notes. Furthermore, clinicians require interpretable rationales—specific clinical themes or "topics"—to validate high-risk predictions.
Traditional methods often utilize Latent Dirichlet Allocation (LDA) to extract topics, which are then used as features for a separate classifier. These "two-stage" processes are suboptimal because topic discovery is not guided by the final prediction task.
Methods
We propose three joint, end-to-end neural network architectures that integrate an LSTM for temporal modeling with specialized topic-modeling layers.
As shown in Figure 5, the architectures vary in how they process the bag-of-words (BoW) representation ($x^{(t)}$) into a latent topic vector ($z^{(t)}$):
- LSTM+E (Encoder): Encodes the BoW directly into a latent vector for the LSTM.
- LSTM+E+D (Encoder-Decoder): Adds a Decoder to reconstruct the original BoW from the latent vector, acting as a regularizer to ensure cohesive topic representations.
- LSTM+E+T+D (Encoder-Transcoder-Decoder): Introduces a "Transcoder" layer and L1 regularization to mimic the sparsity of traditional LDA.
The models are trained to minimize a joint loss function comprising mortality prediction error and document reconstruction error.
Experimental Setup
The models were evaluated using the MIMIC-III dataset, a large-scale database of de-identified ICU patient records. The study focused on textual clinical notes, excluding discharge summaries to prevent label leakage. The final dataset included 36,218 patients, with text preprocessed into BoW representations using the top 500 tf-idf words.
Key Results
The joint models consistently outperformed the LSTM+LDA baseline across multiple mortality prediction tasks:
- Predictive Accuracy: The LSTM+E model improved AUC by approximately +4% for in-hospital mortality and +7% for 1-year post-discharge mortality compared to the baseline.
- Temporal Robustness: LSTM-based models maintained stable performance over longer patient stays, whereas traditional SVM-based models often saw performance degradation.
- Latent Space Separation: Joint training allowed the model to learn topics specifically predictive of outcomes, resulting in clearer clustering of "died" versus "survived" instances in the latent representation.
Discussion and Limitations
Despite improved predictive performance, interpretability remains a challenge. The learned topics were often noisy, containing medical jargon and typos (e.g., "evenign"). Additionally, aggregating notes into 12-hour windows may have mixed disparate clinical contexts, such as nursing and radiology reports. Future work will focus on more aggressive text cleaning and finer-grained temporal modeling to improve topic quality and clinical utility.
Summary updated