What’s Going On With Me and How Can I Better Manage My Health? The Potential of GPT-4 to Transform Discharge Letters Into Patient-Centered Letters to Enhance Patient Safety: Prospective, Exploratory Study

Felix Eisinger, Friederike Holderried, Moritz Mahling, Christian Stegemann–Philipps, Anne Herrmann‐Werner, Eric Nazarenus, Alessandra Sonanini, Martina Guthoff, Carsten Eickhoff, Martin HolderriedJournal of Medical Internet Research 2024 · DOI

Introduction

Hospital discharge letters are a cornerstone of patient safety, serving as the primary source of instructions for post-hospital care. However, these documents are typically written by clinicians for other clinicians, resulting in a "paternalistic" style filled with dense medical jargon and abbreviations. This communication barrier is a significant factor in patient non-adherence; nearly 50% of patients with chronic diseases fail to follow their prescribed therapies, often because they do not understand the "why" or "how" of their care. While patient-centered letters are known to improve outcomes, the time required to draft them manually is a major hurdle for busy medical staff. This study explores whether Large Language Models (LLMs), specifically GPT-4, can automate the transformation of clinical summaries into accessible, actionable patient guides.

Methods

Researchers conducted a prospective, exploratory study using a "gold standard" approach to evaluate GPT-4's performance. They developed three high-fidelity synthetic discharge letters covering common conditions: arterial hypertension (AHT), type 2 diabetes (DM), and diabetic kidney disease (DKD).

To measure the effectiveness of the transformation, the team defined 72 specific "learning objectives" (LOs)—safety-relevant pieces of information essential for patient care. These LOs were categorized using Bloom’s Revised Taxonomy into two cognitive levels: Remember (factual recall, such as medication names) and Understand (comprehending the underlying reasoning, such as why a medication is necessary for kidney protection).

Figure 1: Study design and multi-stage prompt chaining workflow for transforming discharge letters.

The transformation process utilized a multi-stage prompt chaining pipeline rather than a single prompt. This included data extraction into structured JSON, general summary generation, action point extraction, and adversarial prompting specifically designed to simplify remaining jargon to an 8th-grade reading level. Finally, two experienced clinicians reviewed 860 generated sentences for medical accuracy and patient-centricity.

Key Results

GPT-4 demonstrated a strong ability to improve the accessibility of medical information, though some safety gaps remained:

  • Information Transfer: The AI successfully captured 78% (56/72) of the safety-relevant learning objectives. However, 15% (11/72) were missed in the majority of generated letters.
  • The Cognitive Gap: Analysis revealed that GPT-4 was significantly better at transferring "Remember" level objectives than "Understand" level ones. Most omissions occurred in the "prevention of complications" field, where the AI struggled with complex, multi-dependency logical structures.
  • Medical Accuracy: 88% of generated sentences were medically correct. Interestingly, 71% of the errors identified were "hallucinations"—information the AI added itself—rather than misinterpretations of the source text.
  • Readability and Jargon: The transformation resulted in a massive leap in readability. Flesch Reading Ease scores moved from "difficult" (~40) to "standard" (~60). Medical terms were reduced from 60% to 15% of the text, and 92% of the remaining terms were accompanied by explanations.

Discussion and Conclusion

GPT-4 shows immense potential for transforming clinical communication into patient-centered narratives. It excels at "soft" communication—addressing the patient directly and simplifying complex terminology. However, the 15% omission rate for safety-critical information indicates that the technology is not yet ready for unsupervised clinical use.

Future improvements may lie in advanced prompt engineering, such as "Chain-of-Thought" prompting to help the AI better link actions to medical reasoning. For now, the researchers suggest a "human-in-the-loop" model where clinicians use AI-generated drafts as a starting point, performing a final safety check before the letters reach the patient. This approach could significantly reduce the administrative burden on doctors while substantially improving patient understanding and safety.

Summary updated

Abstract

Our study demonstrates that GPT-4 has the potential to transform discharge letters into more patient-centered communication. While the readability and patient centricity of the transformed letters are well-established, they do not fully address all patient safety-relevant information, resulting in the omission of key aspects. Further optimization of prompt engineering may help address this issue and improve the completeness of the transformation.
BibTeX
@article{eisinger2024whats,
  author = {Felix Eisinger and Friederike Holderried and Moritz Mahling and Christian Stegemann–Philipps and Anne Herrmann‐Werner and Eric Nazarenus and Alessandra Sonanini and Martina Guthoff and Carsten Eickhoff and Martin Holderried},
  title = {{What’s Going On With Me and How Can I Better Manage My Health? The Potential of GPT-4 to Transform Discharge Letters Into Patient-Centered Letters to Enhance Patient Safety: Prospective, Exploratory Study}},
  journal = {Journal of Medical Internet Research},
  year = {2024},
  doi = {10.2196/67143},
}
Figure