ICE-NODE: Integration of Clinical Embeddings with Neural Ordinary Differential Equations

Asem Alaa, Erik Mayer, Mauricio Barahona

Abstract

Predicting disease progression from Electronic Health Records (EHR) is challenging due to the irregular timing of clinical events. Traditional models often treat patient histories as discrete sequences, failing to capture the continuous nature of biological health. This paper introduces ICE-NODE (Integration of Clinical Embeddings with Neural Ordinary Differential Equations), a framework that models patient health trajectories as continuous-time functions. By integrating semantic clinical embeddings with Neural ODEs, ICE-NODE effectively handles irregular sampling and improves prediction accuracy for disease onset, particularly for rare conditions.

1. Introduction

The ability to predict disease onset before it reaches a critical stage is a primary goal of modern healthcare informatics. Electronic Health Records (EHRs) provide a rich longitudinal history of patient interactions, yet they are notoriously difficult to model. Most existing machine learning architectures, such as Recurrent Neural Networks (RNNs) and Transformers, treat patient data as discrete sequences of events. However, a patient is a dynamic system evolving in continuous time, not a series of equally spaced tokens. ICE-NODE addresses this discrepancy by shifting the modeling paradigm toward continuous-time dynamics.

2. The Challenge of Irregular Time in EHRs

Disease Progression Modeling (DPM) tools typically capture the order of clinical events (e.g., a cough followed by a fever). However, the temporal gap between these events is often as informative as the events themselves. In an EHR, the interval between hospital visits might vary from hours to years.

Standard models often discard this temporal nuance by binning time into fixed intervals or treating gaps as uniform steps. This approach loses critical clinical context; for instance, a patient returning to the clinic after 48 hours indicates a very different trajectory than a routine six-month follow-up. ICE-NODE seeks to model the underlying "latent" health state of the patient as it evolves continuously, allowing for a more granular understanding of disease velocity.

3. Methodology

The ICE-NODE framework is built around a novel Integrate-Update cycle designed to bridge the gap between discrete clinical observations and continuous biological evolution.

3.1 Neural ODEs for Integration

Instead of discrete state transitions, ICE-NODE utilizes a Neural ODE to define the derivative of the patient’s hidden state. Between clinical visits, an ODE solver integrates this state, allowing the model to estimate the patient’s health at any arbitrary point in time. This ensures that the model accounts for the exact duration of time elapsed between observations.

3.2 Clinical Embeddings and GRAM

To manage the high dimensionality of ICD-9/10 medical codes, ICE-NODE employs semantic embeddings. The researchers utilized GRAM (Graph-based Attention Model), which leverages medical ontologies. By incorporating the hierarchy of diseases, the model can better represent rare clinical codes by associating them with their more common "ancestor" categories in the medical taxonomy.

3.3 Hybrid Hidden State and Regularization

The patient representation is bifurcated into two components:

  1. Memory State: Captures long-term clinical history.
  2. Embedding State: Captures the most recent clinical information.

To ensure the learned trajectories are physically plausible and not overly erratic, the authors implemented a higher-order derivative regularization term, forcing the model toward "smooth" transitions in the latent space.

4. Experimental Setup

The framework was evaluated using two gold-standard datasets:

  • MIMIC-III: A cohort of 4.4K patients with multiple admissions.
  • MIMIC-IV: A larger, more recent dataset, providing a cohort of over 70K patients.

Clinical codes were mapped to the Clinical Classifications Software (CCS) scheme to reduce dimensionality while preserving clinical relevance.

5. Key Results and Insights

5.1 Superior Predictive Accuracy

ICE-NODE consistently outperformed established baselines, including RETAIN and standard GRU-based models. It showed significant improvements in "Top-15" prediction accuracy, particularly in identifying the correct diagnosis within the model's primary suggestions.

5.2 Performance on Rare Conditions

A notable strength of ICE-NODE is its performance on infrequent clinical codes. In the 0-20% and 20-40% frequency quantiles, ICE-NODE variants were the top performers. By modeling continuous trajectories, the model develops a more robust "intuition" for how rare complications emerge from common symptoms.

5.3 Continuous Risk Trajectories

Unlike traditional RNNs that provide predictions only at the moment of a visit, ICE-NODE generates a continuous risk curve. This allows clinicians to visualize the probability of a condition (e.g., acute renal failure) rising over time, even during periods when the patient is not physically present in the hospital.

6. Discussion and Future Work

While ICE-NODE represents a significant step forward, it faces challenges regarding sensitivity to noise. Because Neural ODEs seek smooth paths, mislabeled codes or incorrect timestamps can disrupt the solver. Additionally, training requires a dual-learning-rate strategy to balance ODE dynamics with the rest of the network architecture.

Future research will focus on causal inference—determining not just what will happen, but why. By analyzing how specific treatments "bend the curve" of a continuous disease trajectory, ICE-NODE could eventually serve as a tool for personalized intervention planning.

7. Conclusion

ICE-NODE demonstrates that treating time as a continuous feature rather than a nuisance leads to more accurate and clinically meaningful disease progression models. By integrating semantic clinical knowledge with the mathematical rigor of Neural ODEs, the framework provides a more faithful representation of the pulse of human health.

Summary updated

BibTeX
@inproceedings{alaa2022ice,
  author = {Asem Alaa and Erik Mayer and Mauricio Barahona},
  title = {{ICE-NODE: Integration of Clinical Embeddings with Neural Ordinary Differential Equations}},
  booktitle = {MLHC},
  year = {2022},
  url = {https://openreview.net/forum?id=8zmpZoPusU},
}
Figure