ROAD2H: Development and evaluation of an open‐source explainable artificial intelligence approach for managing co‐morbidity and clinical guidelines

Jesús Domínguez, Denys Prociuk, Branko Marović, Kristijonas Čyras, Oana Cocarascu, Francis Ruiz, Ella Mi, Emma Mi, Christian Ramtale, Antonio Rago, Ara Darzi, Francesca Toni, Vasa Ćurčin, Brendan DelaneyLearning 2023 · DOI

TL;DR

This paper introduces ROAD2H, an open-source, standards-based approach to building explainable AI for clinical decision support (CDS) that handles patient co-morbidities. By integrating an ontological model (TMR) with argumentation-based reasoning (ABA+G) and modern healthcare standards (FHIR, CDS Hooks), it aims to provide conflict-free, explainable recommendations within existing electronic health records (EHRs).

What Problem Does This Paper Address?

Clinical guidelines are essential for standardized care, but real-world patients often have multiple chronic conditions (multimorbidity). This creates complex scenarios where guidelines can conflict, leading to "alert fatigue" and potential patient safety issues. Existing CDS systems can be proprietary, difficult to integrate, and lack transparency in their recommendations. The authors tackle the challenge of creating a CDS system that can:

  • Integrate multiple clinical guidelines: Handle situations where a patient has more than one condition.
  • Resolve guideline conflicts: Identify and manage contradictory recommendations arising from co-morbidities.
  • Provide explainable recommendations: Clearly articulate why a particular recommendation is made.
  • Be interoperable: Seamlessly integrate with existing EHR systems using open standards, especially in low- and middle-income countries (LMICs) where proprietary solutions are often inaccessible.

Methods

The core of the ROAD2H approach lies in combining several key technologies:

  • Transition-based Medical Recommendation (TMR) model: An ontological framework used to represent clinical guidelines. It defines care actions and their expected effects on measured patient properties, allowing for the representation of guideline logic.
  • Assumption-Based Argumentation with Preferences and Goals (ABA+G): This computational argumentation framework is used to reason over potential conflicts between TMR-represented recommendations. It allows for the representation of rules, defeasible assumptions, preferences, and goals to determine the "winning" arguments and thus the most appropriate recommendations.
  • Microservice Architecture: The system is built using a modular microservice architecture. This allows for independent development, deployment, and scaling of different components.
  • Open Standards (HL7 FHIR and CDS Hooks): The system leverages HL7 FHIR (Fast Healthcare Interoperability Resources) for data exchange and CDS Hooks for integrating CDS services into EHRs. This ensures interoperability and makes the approach more widely applicable.

The system works by receiving a CDS request from an EHR, processing patient data, identifying relevant TMR-based guidelines, detecting potential interactions using ABA+G, resolving conflicts, and returning personalized, explainable care plan proposals embedded in a CDS card.

Datasets

The paper describes an evaluation conducted using simulated patient cases.

  • Simulated Case Vignettes: 20 simulated cases were created to represent patients with Chronic Obstructive Pulmonary Disease (COPD) and co-morbidities (cardiovascular disease or chronic kidney disease). These cases were populated into a commercial EHR system (Heliant) for evaluation.
  • GOLD Guideline: The 2017 Global Initiative for Chronic Obstructive Lung Disease (GOLD) guideline was used as the primary source for computable guidelines.

No new benchmark datasets were introduced; rather, existing clinical guidelines and simulated patient data were used for evaluation.

Key Insights and Ideas

  • Argumentation as a Conflict Resolution Engine: The paper highlights how argumentation, specifically ABA+G, is a powerful tool for managing the inherent conflicts that arise when combining multiple clinical guidelines for patients with co-morbidities. It moves beyond simple rule-based systems to a more nuanced reasoning process.
  • Ontology-Driven Explainability: By using an ontological framework like TMR, the system can generate explanations that are not just textual outputs but are grounded in the structured knowledge representation of the guidelines. This makes the explanations more robust and potentially easier to translate.
  • The Power of Open Standards for LMICs: The emphasis on HL7 FHIR and CDS Hooks is crucial. It demonstrates a path towards implementing advanced CDS systems in resource-constrained settings, democratizing access to sophisticated clinical decision support.
  • Flexibility of Microservices: The microservice architecture allows for the system to be extended to other diseases or specialties by simply formalizing new guidelines within the TMR framework and registering new CDS hooks.

Key Results

The evaluation involved five pulmonologists assessing 20 simulated cases each.

  • High Agreement with GOLD Group Assignment: Pulmonologists agreed 97% of the time with the GOLD group assigned by the CDSS.
  • High Agreement with Care Plans: 98% of the time, pulmonologists agreed with one of the proposed COPD care plans.
  • Qualitative Feedback: Clinicians found the system's principles of explainable argumentation favorable. They suggested including more co-morbidities and customizing the level of explanation based on expertise in future versions.
  • Usability: The system was generally perceived as "clear and concise." While some minor technical issues related to data entry were noted, they were easily resolved.
  • Treatment Options: Clinicians appreciated the wide range of treatment options offered, including alternative therapeutic options and non-medical recommendations.

Limitations and Future Work

  • Time-Consuming Guideline Representation: Representing guidelines in TMR is currently a manual and time-consuming process, requiring expertise from clinicians and knowledge engineers. Future work could explore semi-automation using natural language processing and knowledge graphs.
  • Limited Scope of Evaluation: The evaluation was limited to interactions within a single guideline (COPD) due to the nature of the clinical use case and COVID-19 restrictions. Future work needs to explore the system's robustness with multiple interacting guidelines.
  • Preference and Goal Integration: The current evaluation omitted the integration of prior patient preferences and overall clinical goals (e.g., cost-effectiveness) due to EHR vendor restrictions. Future work should re-incorporate these aspects.
  • Temporal Reasoning and Dosing: The current TMR model is suitable for cyclic workflows but needs enhancement for temporal reasoning, drug dosing, and more complex guideline flow control.
  • Explanation Granularity: While ABA+G provides explanations, future work could explore more interactive forms of explanation, allowing users to ask questions.
  • Expansion to Other Diseases: The authors envision expanding the system to manage other chronic diseases by formalizing their respective guidelines and deploying additional TMR-based CDS services.

Summary updated

Abstract

An ontological model provided a flexible means of providing argumentation and explainable artificial intelligence for a long-term condition. Extension to other guidelines and multiple co-morbidities is needed to test the approach further.
BibTeX
@inproceedings{dominguez2023road,
  author = {Jesús Domínguez and Denys Prociuk and Branko Marović and Kristijonas Čyras and Oana Cocarascu and Francis Ruiz and Ella Mi and Emma Mi and Christian Ramtale and Antonio Rago and Ara Darzi and Francesca Toni and Vasa Ćurčin and Brendan Delaney},
  title = {{ROAD2H: Development and evaluation of an open‐source explainable artificial intelligence approach for managing co‐morbidity and clinical guidelines}},
  booktitle = {Learning},
  year = {2023},
  doi = {10.1002/lrh2.10391},
}
Figure