Abstract
Electronic Health Records (EHR) contain a wealth of heterogeneous data, including structured codes, numerical lab results, and unstructured clinical notes. Effectively modeling this multi-modal information is challenging due to modality silos and the scarcity of labeled data. We propose the Mixed Pooling Multi-View Attention Autoencoder (MPVAA), an unsupervised framework that learns comprehensive patient embeddings. MPVAA utilizes graph embeddings for individual data views, a novel stochastic mixed pooling strategy for robust feature extraction, and a multi-view attention decoder to fuse diverse information streams. Our experiments on the MIMIC-III dataset show that MPVAA significantly outperforms existing baselines in clinical outcome prediction tasks.
Introduction
The digital transformation of healthcare has resulted in massive repositories of Electronic Health Records (EHR). These records are inherently multi-modal, capturing a patient's journey through demographics, laboratory tests, medications, and physician narratives. However, most current predictive models focus on a single data type or require extensive manual labeling, which is both costly and difficult to scale.
To address these challenges, we introduce the Mixed Pooling Multi-View Attention Autoencoder (MPVAA). MPVAA is designed to learn "patient embeddings"—dense vector representations—in an unsupervised manner. By treating different EHR data types as distinct "views," the model captures complementary information and non-linear interactions that are often missed by traditional methods.
Methodology
The MPVAA architecture follows an encoder-decoder structure optimized for heterogeneous healthcare data. It is built upon three primary components:
Inner-View Graph Embeddings
For each patient, the model constructs specific graphs for three distinct views: demographics (dem), laboratory results (lab), and clinical notes (notes). Graph Autoencoders (GAE) are employed to learn the relationships between medical concepts within these specific contexts, allowing the model to understand the underlying structure of the data before fusion.
Stochastic Mixed Pooling
A key innovation of MPVAA is its stochastic mixed pooling strategy. While standard models typically use either mean pooling (to capture global trends) or max pooling (to identify the most salient features), MPVAA combines both. During training, a random weight ($\lambda$) is used to balance these two pooling methods. This stochasticity forces the encoder to produce more robust and expressive latent representations, preventing over-reliance on a single feature extraction mode.
Multi-View Attention Decoder
The reconstruction phase utilizes a multi-view attention mechanism. Instead of processing data streams in isolation, the decoder reconciles interactions between different views. For instance, it can learn how a specific demographic risk factor correlates with a particular lab result trend, leading to a more holistic reconstruction of the patient's state.
Experimental Evaluation
The model was evaluated using the MIMIC-III dataset, comprising de-identified clinical records from over 40,000 critical care patients. The study focused on a cohort of 7,499 patients with multiple visits, extracting ICD-9 codes, NDC medication codes, and discharge summaries.
Clinical Outcome Prediction
MPVAA was tested on two primary tasks:
- Heart Failure (HF) Prediction: The model achieved a 13% performance gain in AUC-ROC and Accuracy compared to specialized baselines like Med2Vec and GRAM.
- Sequential Disease Prediction: MPVAA outperformed all baselines in predicting future diagnosis codes, showing a 3.5% improvement in Normalized Cumulative Discounted Gain (NCDG).
Ablation Studies
Ablation experiments confirmed the necessity of the model's core components. Removing either the mixed pooling strategy or the multi-view attention mechanism resulted in a significant drop in performance, validating the integrated approach to multi-modal representation learning.
Discussion and Conclusion
MPVAA demonstrates the power of attention-based unsupervised learning in healthcare. By moving away from recurrent architectures, the model avoids the bias toward recent events and can effectively "look back" across a patient's entire history. The use of multiple views allows the model to treat data heterogeneity as a feature rather than a hurdle, capturing nuances across different clinical modalities.
While the model shows strong performance, future work will address the "black box" nature of deep learning in clinical settings by improving the interpretability of attention weights. Additionally, while the current study focused on the MIMIC-III dataset, the framework's generalizability to other domains with heterogeneous data remains an area for further exploration.
Summary updated