Feasibility and impact of a computerised clinical decision support system on investigation and initial management of new onset chest pain: a mixed methods study

Rachel Johnson, Maggie Evans, Helen Cramer, Kristina Bennert, Richard Morris, Sandra Eldridge, Katy Juttner, Mohammed J. Zaman, Harry Hemingway, Spiros C. Denaxas, Adam Timmis, Gene FederBMC Medical Informatics Decis. Mak 2015 · DOI

TL;DR

This study investigated the feasibility and impact of a computerized clinical decision support system (CDSS) designed to aid clinicians in managing new-onset chest pain. Despite high reported usage, the CDSS was largely used for "decision feedback" rather than real-time guidance, with clinicians often consulting it after patient interactions. The study highlights significant barriers to effective CDSS integration in clinical practice, suggesting that current systems may not be feasible or impactful without substantial redesign.

What Problem Does This Paper Address?

Clinical decision support systems (CDSSs) hold promise for improving healthcare by providing evidence-based recommendations to clinicians. However, understanding the factors that influence their effectiveness, particularly in real-world clinical settings, remains a challenge. This paper addresses the gap in understanding the practical feasibility and impact of a CDSS designed to support the investigation and initial management of patients presenting with new-onset chest pain. Chest pain is a common and critical symptom, and accurate diagnosis and timely management are crucial to prevent adverse cardiovascular events. Existing guidelines exist, but their application in busy clinical environments can be complex. This study aims to explore how a CDSS, based on national guidelines, is actually used by clinicians and what impact it has on their decision-making and patient management.

Methods

The study employed a mixed-methods approach, combining a before-and-after quantitative study with qualitative data collection.

  • Quantitative Component: A before-and-after study was conducted in three chest pain clinics. Data on patient investigations and medication were collected for patients seen before the introduction of the CDSS and for those seen after. Concordance between CDSS recommendations and actual clinical practice was assessed.
  • Qualitative Component: This involved observations of clinical practice, semi-structured interviews with clinicians, and a focus group. These methods were used to understand the context of CDSS use, clinicians' perceptions, and the barriers and facilitators to its integration into their workflow. The qualitative data were analyzed thematically.

The CDSS itself was developed based on NICE guidelines for chest pain and incorporated a system for calculating a patient's pre-test probability for coronary heart disease, guiding subsequent investigation and medication recommendations.

Datasets

  • Patient Data: Data were collected from 285 patients attending chest pain clinics (106 before the OMA program and 179 after). This included demographic information, cardiovascular risk factors, resting ECG, typicality of chest pain, and current medications.
  • Clinical Records: Information on cardiac investigations performed within six months of presentation and medication prescriptions at six months was extracted from electronic primary care records and hospital records.
  • Observational and Interview Data: Detailed field notes and audio recordings from 40 observed consultations, 5 clinician interviews, and 1 focus group provided qualitative insights.

Key Insights and Ideas

  • "Decision Feedback" vs. "Decision Support": Clinicians primarily used the CDSS as a tool for "decision feedback" – confirming their existing decisions – rather than as a genuine "decision support" system that actively guided their real-time choices.
  • Post-Consultation Use: In a majority of observed cases, the CDSS was consulted after the patient had left the room, indicating it was not integrated into the dynamic decision-making process during the consultation.
  • Data Entry Challenges: Clinicians faced difficulties in accurately and efficiently entering patient data into the CDSS, particularly in categorizing chest pain as "typical," "atypical," or "non-cardiac," which often required them to adapt their usual practice or even manipulate responses to fit the system's algorithm.
  • Risk Factor Simplification: The binary (yes/no) input for cardiovascular risk factors was seen as a significant limitation, failing to capture the nuances of severity that clinicians used in their own risk assessments.
  • Embedded Clinical Expertise: Clinicians consistently prioritized their clinical expertise and judgment over CDSS recommendations, especially when the CDSS suggested a different course of action than what their experience dictated.
  • Structural and Practical Barriers: Beyond data entry, issues like the unavailability of certain investigations, lack of prescribing privileges for non-physicians, and the need to duplicate data entry into separate systems hindered effective CDSS use.

Key Results

  • CDSS Usage: The CDSS was used in 86% of consultations where it was available in the "After OMA" group.
  • Investigation Concordance: The proportion of patients receiving recommended investigations was similar before (50%) and after (59%) the OMA program, suggesting no significant impact on investigation choice.
  • Medication Concordance: The study found little evidence of impact on medication prescribing.
  • Clinician Disagreement: Clinicians disagreed with CDSS recommendations in almost half of observed cases, often opting for more investigations than recommended, particularly when pain was categorized as "atypical."
  • Feasibility: The study concluded that the OMA CDSS, in its current form, was not feasible for effective integration into clinical practice.

Limitations and Future Work

  • Small Sample Size: The quantitative study had a relatively small sample size, precluding robust statistical analysis.
  • Censored Follow-up: The six-month follow-up period meant that investigations occurring later might have been missed.
  • Guideline Lag: The CDSS was based on NICE guidance that was published shortly before its implementation, and the entrenched use of exercise ECGs (which the NICE guidance advised against) created a mismatch.

The authors suggest that future CDSS development should focus on:

  • Better Integration: Addressing structural and practical barriers, such as improving data entry and potentially integrating with existing electronic systems.
  • Reflecting Clinical Workflow: Designing systems that more closely align with how clinicians actually assess and manage patients, rather than imposing a rigid new workflow.
  • Enhanced Training and Support: Providing more intensive training and ongoing monitoring for CDSS users.
  • Patient and Practitioner Guidance: Developing clearer guidance for both clinicians and patients on the use and interpretation of CDSS recommendations.

Summary updated

BibTeX
@article{johnson2015feasibility,
  author = {Rachel Johnson and Maggie Evans and Helen Cramer and Kristina Bennert and Richard Morris and Sandra Eldridge and Katy Juttner and Mohammed J. Zaman and Harry Hemingway and Spiros C. Denaxas and Adam Timmis and Gene Feder},
  title = {{Feasibility and impact of a computerised clinical decision support system on investigation and initial management of new onset chest pain: a mixed methods study}},
  journal = {BMC Medical Informatics Decis. Mak.},
  year = {2015},
  doi = {10.1186/s12911-015-0189-8},
}
Figure