TrajVis: a visual clinical decision support system to translate artificial intelligence trajectory models in the precision management of chronic kidney disease

Zuotian Li, Xiang Liu, Ziyang Tang, Pengyue Zhang, Nanxin Jin, Michael T. Eadon, Qianqian Song, Yingjie Chen, Jing SuJ. Am. Medical Informatics Assoc 2024

TL;DR

This paper introduces TrajVis, a novel visual analytics system designed to bridge the gap between complex AI models predicting chronic kidney disease (CKD) progression and their practical use by clinicians. By translating AI-generated patient trajectories into intuitive visualizations, TrajVis aims to empower healthcare professionals with actionable insights for personalized patient management.

What Problem Does This Paper Address?

Managing chronic diseases like Chronic Kidney Disease (CKD) is challenging due to their complexity and heterogeneity. Clinicians typically rely on a limited set of indicators, which can lead to delayed detection of progression or overdiagnosis. While Artificial Intelligence (AI) models show promise in predicting disease trajectories by uncovering complex patterns in longitudinal electronic medical records (EMR), their "black-box" nature often hinders clinical adoption. Clinicians may lack the time or expertise to interpret these models, leading to a disconnect between AI advancements and real-world healthcare applications. This paper addresses this gap by developing a system that visualizes and explains AI-driven CKD progression trajectories, making them accessible and useful for clinical decision-making.

Methods

The core of TrajVis is built around the DEPOT (DisEase PrOgression Trajectory) graph AI model, which infers CKD progression trajectories from EMR data. The system employs a multi-component architecture:

  • Data Component: Harmonized EMR data from the Atrium Health Wake Forest Baptist Translational Data Warehouse and the Indiana Network for Patient Care.
  • AI/ML Model Component: The DEPOT graph AI model for trajectory inference. Other models like Group-Based Trajectory Modeling (GBTM) can also be integrated.
  • Patient Trajectory Component: Visualizes patient visits projected into a latent space, representing three established CKD trajectories (healthy, late progression, early progression) using UMAP.
  • Interpretation Component: Provides explanations for AI predictions, identifying key clinical indicators (predictors and markers) that contribute to a patient's trajectory and assessing the likelihood of a patient belonging to each trajectory.

The system's frontend is built with React and Echart, communicating with the backend via a RESTful API.

Datasets

The study utilizes de-identified longitudinal EMR data from two sources:

  • Atrium Health Wake Forest Baptist Translational Data Warehouse (WF-TDW): Data from 01/01/1993 to 06/30/2018.
  • Indiana Network for Patient Care (INPC): Data from 01/01/2016 to 05/31/2022.

For demonstration purposes, synthetic longitudinal EMR records were generated, comprising 79,434 patients, 508,731 encounters, and 1,057,253 records. Dates were randomly shifted by up to 6 months, and 10% of encounter data was replaced with similar encounters from different patients to create synthetic data.

Key Insights and Ideas

  • Bridging the AI-Clinical Divide: The primary insight is that effective visualization is crucial for translating complex AI predictions into clinically actionable insights. TrajVis demonstrates that by making AI outputs interpretable, clinicians can gain confidence and utility from these models.
  • Multi-faceted Visualization: TrajVis employs four distinct views (Patient View, Trajectory View, Clinical Indicator View, Analysis View) to provide a comprehensive understanding of patient data, AI predictions, and contributing factors. This layered approach caters to different aspects of clinical decision-making.
  • Latent Space Visualization: Projecting patient visit data into a 2D latent space using UMAP allows for intuitive visualization of disease progression trajectories and their relationships with clinical features, even when the underlying AI model operates in a higher-dimensional space.
  • Explainable AI for Healthcare: The system emphasizes explaining why an AI model makes a certain prediction by highlighting the specific clinical indicators that influence the trajectory, which is vital for clinician trust and adoption.

Key Results

  • System Functionality: TrajVis successfully integrates AI models (specifically DEPOT) to visualize CKD progression trajectories and their contributing clinical factors.
  • Case Study Effectiveness: A case study with two typical patients demonstrated TrajVis's ability to help clinicians identify different CKD progression trajectories, investigate medical history, and understand associated clinical features. The system was found to be a convenient panel for assessing longitudinal trends and identifying risk factors beyond single indicators like eGFR.
  • User Survey Satisfaction: A user survey of 17 participants (clinicians and data scientists) indicated positive reception. 12 participants were very satisfied (score >= 6) with the integration of AI. Users with computational expertise found TrajVis intuitive, helpful, and easy, while even those without such background reported comfort in using it for AI integration into clinical care.

Limitations and Future Work

  • Limitations:
    • The study used synthetic EMR data for demonstration purposes, which may not fully capture the nuances of real-world data.
  • While the user survey showed positive results, it involved a relatively small sample size.
  • Future Work:
    • The authors suggest that TrajVis can be extended to incorporate other trajectory learning models and potentially other chronic diseases.
  • Further validation with larger, real-world datasets and a broader range of clinical users is warranted.
  • The potential for TrajVis to be used for screening populations for renal clinic referral and identifying patients at high disease progression risk that might escape common clinical surveillance was highlighted by users.

Summary updated

Abstract

Objective: Our objective is to develop and validate TrajVis, an interactive tool that assists clinicians in using artificial intelligence (AI) models to leverage patients' longitudinal electronic medical records (EMR) for personalized precision management of chronic disease progression. Methods: We first perform requirement analysis with clinicians and data scientists to determine the visual analytics tasks of the TrajVis system as well as its design and functionalities. A graph AI model for chronic kidney disease (CKD) trajectory inference named DEPOT is used for system development and demonstration. TrajVis is implemented as a full-stack web application with synthetic EMR data derived from the Atrium Health Wake Forest Baptist Translational Data Warehouse and the Indiana Network for Patient Care research database. A case study with a nephrologist and a user experience survey of clinicians and data scientists are conducted to evaluate the TrajVis system. Results: The TrajVis clinical information system is composed of four panels: the Patient View for demographic and clinical information, the Trajectory View to visualize the DEPOT-derived CKD trajectories in latent space, the Clinical Indicator View to elucidate longitudinal patterns of clinical features and interpret DEPOT predictions, and the Analysis View to demonstrate personal CKD progression trajectories. System evaluations suggest that TrajVis supports clinicians in summarizing clinical data, identifying individualized risk predictors, and visualizing patient disease progression trajectories, overcoming the barriers of AI implementation in healthcare. Conclusion: TrajVis bridges the gap between the fast-growing AI/ML modeling and the clinical use of such models for personalized and precision management of chronic diseases.
BibTeX
@article{li2024trajvis,
  author = {Zuotian Li and Xiang Liu and Ziyang Tang and Pengyue Zhang and Nanxin Jin and Michael T. Eadon and Qianqian Song and Yingjie Chen and Jing Su},
  title = {{TrajVis: a visual clinical decision support system to translate artificial intelligence trajectory models in the precision management of chronic kidney disease}},
  journal = {J. Am. Medical Informatics Assoc.},
  year = {2024},
}
Figure