Preliminary evaluation of the Chest Pain Dashboard, a FHIR-based approach for integrating health information exchange information directly into the clinical workflow.

Titus Schleyer, Saurabh Rahurkar, Allissia M Baublet, Matthias Kochmann, Xia Ning, Douglas K. Martin, Keith W. Kelley, Jason T. SchafferPubMed 2019

TL;DR

This paper introduces a "Chest Pain Dashboard" that integrates Health Information Exchange (HIE) data directly into Electronic Health Records (EHRs) using the FHIR standard. Preliminary results show a significant increase in HIE usage in the intervention Emergency Department (ED) after its implementation, suggesting FHIR-based solutions can improve clinical workflow and data accessibility.

What Problem Does This Paper Address?

Despite significant efforts, integrating clinical data from HIEs and EHRs remains a major challenge, largely due to inefficiencies and workflow interruptions. Clinicians often have to access HIE data through separate systems (like web portals), which disrupts their workflow, increases the number of clicks and cognitive effort required, and ultimately leads to low HIE utilization. A pilot study observed that in busy EDs, HIE was accessed only twice in 70 patient encounters, highlighting the critical need for seamless integration. This paper aims to address this gap by developing and evaluating a novel approach that embeds HIE data directly within the EHR interface.

Methods

The researchers developed a "Chest Pain Dashboard" leveraging the Fast Healthcare Interoperability Resources (FHIR) standard. This dashboard integrates data from the Indiana Network of Patient Care (INPC), a large inter-organizational clinical data repository, directly into the Cerner EHR system used at an Indiana University Health ED. The integration was achieved by building a custom FHIR interface to the INPC (FHIR-on-INPC Server) that exposes selected data resources. Clinicians can launch the dashboard with a single click from within Cerner, and their credentials are automatically passed, eliminating the need for separate logins. The dashboard searches for five key data elements relevant to chest pain patients: last EKG, cardiology note, discharge summary, cardiac catheterization report, and echo/stress/nuclear medicine tests.

The evaluation employed a mixed-methods approach:

  • Quantitative: A quasi-experimental, longitudinal study design using a difference-in-difference (DID) model to assess the impact of the dashboard on HIE usage. This approach compares HIE usage in the intervention ED before and after dashboard implementation against control EDs.
  • Qualitative: Semi-structured interviews and a survey (based on the Post-Study e-Health Usability Questionnaire - PSHUQ) to gather user feedback on system usefulness, information quality, system efficiency, and potential improvements.

Datasets

  • Indiana Network of Patient Care (INPC): A large inter-organizational clinical data repository used as the source for HIE data.
  • Cerner EHR System: The EHR system used at Indiana University Health Methodist Hospital, where the dashboard was implemented.
  • Encounter Data: Logged data from all emergency department encounters at IUH system hospitals during the study period (September 2017 to March 2018).
  • INPC Usage Logs: Records of user queries for patient clinical information through CareWeb or the Dashboard.
  • Dashboard Logs: Files generated by the Dashboard itself to track usage.

No new benchmarks were introduced; the study focused on evaluating the impact of the implemented dashboard on existing HIE usage patterns.

Key Insights and Ideas

  • FHIR as a Game Changer for Interoperability: The paper strongly advocates for FHIR as a key enabler for seamless integration of HIE data into EHRs, moving beyond traditional, cumbersome methods.
  • Workflow Integration is Paramount: The success of HIE adoption hinges on minimizing workflow disruptions. Embedding data directly within the EHR, as demonstrated by the dashboard, is crucial.
  • User Experience Drives Adoption: While the idea of integrated HIE data is loved, the implementation needs to be user-friendly. Early versions of the dashboard had issues with response time and errors, which significantly impacted user perception.
  • Targeted Data Improves Utility: Focusing on specific, high-value data elements (like those for chest pain) makes the integrated information more relevant and actionable for clinicians.
  • Mixed-Methods Evaluation is Essential: Combining quantitative data on usage with qualitative feedback on user experience provides a more comprehensive understanding of the technology's impact and areas for improvement.

Key Results

Quantitative:

  • The difference-in-difference analysis showed a significant positive relationship between the Dashboard implementation and HIE usage (Odds Ratio: 1.15; 95% CI: 1.02, 1.30; p=0.026). This suggests that clinicians in the intervention ED used the HIE more after the Dashboard was implemented compared to other EDs.
  • During the study period, the intervention ED had greater INPC usage (10.55% vs 3.2%) compared to other EDs.
  • INPC use increased to a greater degree in the intervention ED in the post-implementation period (56.0% vs 53.5%).
  • In a laboratory experiment during development, the Dashboard reduced the number of clicks from 50 to 6 and search time from 3 minutes to 10 seconds compared to the traditional CareWeb method.

Qualitative:

  • System Usefulness: Overwhelmingly positive, with respondents finding the dashboard easy to use and learn, and appreciating the "one-click" access to needed information.
  • System/Information Quality: Mixed. While some found the information helpful and easy to understand, there were significant issues with error handling, response time, and the completeness of retrieved notes.
  • System Efficiency: Predominantly negative in early versions, with users reporting slow response times (10-30 seconds) and interface freezes. A later version improved this to 10-20 seconds.
  • Potential System Improvements: Strong agreement that the system could be improved, with users suggesting better error handling and more comprehensive data retrieval.

Limitations and Future Work

Limitations:

  • "Moving Target" Evaluation: The study evaluated an early, non-optimized version of the Dashboard, which was later re-designed and re-implemented. This likely biased findings towards the null.
  • Limited Scope: The study focused on a single ED and a specific chief complaint (chest pain), though broader utility was observed.
  • Preliminary Findings: The paper presents preliminary findings, with further analysis planned for other hypotheses (e.g., time spent in EHR, clinical decision-making impact).
  • Instrument Validity: Formal assessment of the quantitative survey instrument's validity and reliability is planned for a future publication.

Future Work:

  • Investigate the impact of the Dashboard on clinician time spent in the EHR and clinical decision-making.
  • Explore the broad-spectrum utility of the Dashboard beyond chest pain chief complaints.
  • Continue to refine the Dashboard based on user feedback, particularly addressing system efficiency and information quality issues.
  • Develop a robust production implementation of the concept, potentially integrating more chief complaints and functionalities.
  • Further investigation is needed to test the hypothesis that clinicians perceive the Dashboard as an efficient way to access useful information across various clinical scenarios.

Summary updated

Abstract

Despite efforts aimed at improving the integration of clinical data from health information exchanges (HIE) and electronic health records (EHR), interoperability remains limited. Barriers due to inefficiencies and workflow interruptions make using HIE data during care delivery difficult. Capitalizing on the development of the Fast Healthcare Interoperability Resource (FHIR) specification, we designed and developed a Chest Pain Dashboard that integrates HIE data into EHRs. This Dashboard was implemented in one emergency department (ED) of Indiana University Health in Indiana. In this paper, we present the preliminary findings from a mixed-methods evaluation of the Dashboard. A difference-in-difference analysis suggests that the ED with the Dashboard implementation resulted in a significant increase in HIE use compared to EDs without. This finding was supported by qualitative interviews. While these results are encouraging, we also identified areas for improvement. FHIR-based solutions may offer promising approaches to encourage greater accessibility and use of HIE data.
BibTeX
@inproceedings{schleyer2019preliminary,
  author = {Titus Schleyer and Saurabh Rahurkar and Allissia M Baublet and Matthias Kochmann and Xia Ning and Douglas K. Martin and Saurabh Rahurkar and Keith W. Kelley and Jason T. Schaffer and Jason T Schaffer},
  title = {{Preliminary evaluation of the <i>Chest Pain Dashboard</i>, a FHIR-based approach for integrating health information exchange information directly into the clinical workflow.}},
  booktitle = {PubMed},
  year = {2019},
}
Figure