Marrying Medical Domain Knowledge With Deep Learning on Electronic Health Records: A Deep Visual Analytics Approach

Rui Li, Changchang Yin, Samuel Yang, Buyue Qian, Ping ZhangJournal of Medical Internet Research 2020 · OpenReview

Abstract

Deep learning models have shown great promise in healthcare, yet their clinical adoption is hindered by a lack of interpretability, the inability to incorporate complex medical domain knowledge, and a lack of interactive exploration tools. This study introduces DG-RNN, a domain-knowledge-guided recurrent neural network for clinical risk prediction, and DG-Viz, an interactive visual analytics system. By integrating a massive medical knowledge graph into the deep learning process, the DG-RNN model achieves a 1.5% improvement in heart failure risk prediction accuracy over state-of-the-art approaches. The DG-Viz system allows clinicians to explore EHR data, visualize knowledge graph connections, and perform "what-if" simulations to interpret model predictions.

Introduction

Electronic Health Records (EHR) contain rich longitudinal data that deep learning models can leverage to predict patient risks. However, three major limitations persist:

  1. The Black Box Problem: Clinicians often cannot interpret the reasoning behind a model's high-risk prediction.
  2. Ignoring Domain Knowledge: Standard models treat medical codes as isolated variables, failing to account for established biological and clinical relationships (e.g., disease causality).
  3. Lack of Interaction: EHR data is complex and abstract; clinicians need tools to interact with models directly to validate findings or explore hypothetical scenarios.

To address these challenges, we present a deep visual analytics approach that marries medical expertise with deep learning.

Methods

DG-RNN Model Architecture

The proposed DG-RNN model processes sequences of medical events. Its core innovation is a graph-based attention mechanism. When processing a specific medical code, the model "attends" to a relevant subgraph of a massive medical knowledge graph (KnowLife), pulling in contextual domain knowledge. To prevent the model from "forgetting" early patient history—a common issue in standard RNNs—we implemented a global max-pooling layer. This layer captures the most significant features across the entire patient timeline, enhancing both prediction performance and interpretability.

DG-Viz System

DG-Viz is an interactive suite designed to support EHR exploration and model interpretation. It consists of four primary components:

  • Distribution View (A): Uses t-SNE to cluster similar patients, allowing for cohort analysis.
  • Demographic Distribution (B): Provides context on race and gender distributions within the selected cohort.
  • Patient History View (C): Displays a timeline of a patient's risk scores, highlighting how specific visits contribute to the overall risk.
  • Knowledge Graph View (D): Visualizes the medical entities and relationships that influenced the model's decision.
The DG-Viz system interface, featuring (A) Distribution View, (B) Demographic Distribution, (C) Patient History, and (D) Knowledge Graph View.

The system also supports "What-If" analysis, enabling clinicians to manually add or remove medical codes from a patient's record to observe real-time changes in predicted risk.

Results

The DG-RNN model was evaluated on a real-world longitudinal EHR dataset of 218,680 patients, specifically focusing on heart failure prediction (554 cases, 1,662 controls).

  • Performance: DG-RNN consistently outperformed baselines (RETAIN, GRAM, KAME), achieving an AUROC of 0.7863—a 1.5% improvement over the previous state-of-the-art.
  • Ablation Study: Removing the medical knowledge graph led to a 2% drop in AUROC, confirming the value of integrating domain knowledge.
  • Clinical Case Study: A medical physician collaborator used DG-Viz to validate the model's findings. The system correctly highlighted atrial fibrillation and cardiac dysrhythmia as primary risk factors for heart failure, aligning with clinical expertise.

Conclusion

DG-Viz demonstrates that combining deep learning with visual analytics can create tools that are both accurate and interpretable. By bridging the gap between "black box" algorithms and clinical domain knowledge, this approach paves the way for more trustworthy and collaborative AI in healthcare. Future work will focus on reducing user cognitive load and explicitly visualizing model uncertainty.

Summary updated

Abstract

Background: Deep learning models have attracted significant interest from health care researchers during the last few decades. There have been many studies that apply deep learning to medical applications and achieve promising results. However, there are three limitations to the existing models: (1) most clinicians are unable to interpret the results from the existing models, (2) existing models cannot incorporate complicated medical domain knowledge (eg, a disease causes another disease), and (3) most existing models lack visual exploration and interaction. Both the electronic health record (EHR) data set and the deep model results are complex and abstract, which impedes clinicians from exploring and communicating with the model directly.Objective: The objective of this study is to develop an interpretable and accurate risk prediction model as well as an interactive clinical prediction system to support EHR data exploration, knowledge graph demonstration, and model interpretation.Methods: A domain-knowledge–guided recurrent neural network (DG-RNN) model is proposed to predict clinical risks. The model takes medical event sequences as input and incorporates medical domain knowledge by attending to a subgraph of the whole medical knowledge graph. A global pooling operation and a fully connected layer are used to output the clinical outcomes. The middle results and the parameters of the fully connected layer are helpful in identifying which medical events cause clinical risks. DG-Viz is also designed to support EHR data exploration, knowledge graph demonstration, and model interpretation.Results: We conducted both risk prediction experiments and a case study on a real-world data set. A total of 554 patients with heart failure and 1662 control patients without heart failure were selected from the data set. The experimental results show that the proposed DG-RNN outperforms the state-of-the-art approaches by approximately 1.5%. The case study demonstrates how our medical physician collaborator can effectively explore the data and interpret the prediction results using DG-Viz.Conclusions: In this study, we present DG-Viz, an interactive clinical prediction system, which brings together the power of deep learning (ie, a DG-RNN–based model) and visual analytics to predict clinical risks and visually interpret the EHR prediction results. Experimental results and a case study on heart failure risk prediction tasks demonstrate the effectiveness and usefulness of the DG-Viz system. This study will pave the way for interactive, interpretable, and accurate clinical risk predictions.J Med Internet Res 2020;22(9):e20645doi:10.2196/20645
BibTeX
@article{li2020marrying,
  author = {Rui Li and Changchang Yin and Samuel Yang and Buyue Qian and Ping Zhang},
  title = {{Marrying Medical Domain Knowledge With Deep Learning on Electronic Health Records: A Deep Visual Analytics Approach}},
  journal = {Journal of Medical Internet Research},
  year = {2020},
  url = {https://openreview.net/forum?id=asuMBxIE5H},
}
Figure