TL;DR
This study demonstrates that GPT-4 can bridge the "health literacy gap" by simplifying complex surgical consent forms from a college-freshman reading level to an 8th-grade level without losing medical or legal accuracy. By introducing a collaborative AI-human expert framework, the researchers show that large language models (LLMs) can generate procedure-specific consents that meet rigorous clinical standards while remaining accessible to the average patient.
What Problem Does This Paper Address?
Informed consent is a cornerstone of medical ethics, yet the documents used to obtain it are often functionally unreadable for the average person. While the average American adult reads at an 8th-grade level, most surgical consent forms are written at a college level or higher. This discrepancy—the "literacy gap"—can lead to poor patient comprehension, increased anxiety, and worse clinical outcomes.
Furthermore, many consent forms are "one-size-fits-all" templates that lack the nuance of specific procedures. Manually rewriting these forms to be both simple and procedure-specific is a massive undertaking requiring significant time from surgeons and legal experts. The authors investigate whether GPT-4 can automate this simplification and generation process while maintaining the "medicolegal" rigor required in a healthcare setting.
Methods
The researchers employed a multi-stage approach to test GPT-4’s capabilities:
- Simplification of Existing Forms: They collected 15 "generic" consent forms from major US academic medical centers. GPT-4 was prompted to convert these texts to an "average American reading level" while preserving content.
- Linguistic Analysis: Readability was quantified using the Flesch-Kincaid Reading Level and Flesch Reading Ease scores. They also used the
koRpuspackage in R and TreeTagger software to analyze "word rarity" against the Corpus of Contemporary American English (COCA), a one-billion-word dataset. - Expert Validation: Three physicians and one malpractice defense attorney independently reviewed the original and simplified forms to ensure no critical information was lost and legal sufficiency was maintained.
- De Novo Generation: GPT-4 was asked to create entirely new, procedure-specific consent forms for five diverse operations (e.g., Deep Brain Stimulation, Appendectomy, Mohs surgery).
- Quality Scoring: These new forms were evaluated using a validated 20-point rubric (the Spatz et al. rubric) and reviewed by subspecialty surgeons for clinical accuracy.
Datasets
- Institutional Consents: 15 generic surgical consent forms from large, geographically diverse US academic medical centers.
- COCA (Corpus of Contemporary American English): Used as a benchmark for word frequency and rarity to determine if the AI was successfully replacing medical jargon with common language.
Key Insights and Ideas
- The "Jargon-to-English" Translation: One of the most effective ways the AI simplified text was by replacing specific medical jargon with descriptive phrases. For example, "respiratory failure" became "losing ability to breathe." This significantly reduced "word rarity" scores.
- Active vs. Passive Voice: The study found a significant reduction in passive voice (from 38.4% to 20.0%). Shifting to active voice makes instructions clearer and more direct, which is a key component of readability that standard metrics often overlook.
- Maintaining Legal Rigor: Perhaps the most surprising insight was that simplification did not result in "dilution." The malpractice attorney confirmed that the simplified forms remained legally sufficient, suggesting that complexity is often a byproduct of tradition rather than a legal necessity.
- Human-in-the-Loop is Essential: The authors emphasize an "AI-human expert collaborative approach." They argue that AI should not work in a vacuum; rather, human experts provide the "social proof" and final verification needed to safely integrate AI into clinical practice.
Key Results
- Readability Leap: GPT-4 improved the average reading level from Grade 13.9 (College Freshman) to Grade 8.9 (8th Grade) ($P=0.004$).
- Efficiency: The median reading time for a consent form dropped from 3.26 minutes to 2.42 minutes ($P<0.001$).
- Perfect Scores for New Forms: The five de novo procedure-specific forms generated by GPT-4 achieved perfect 20/20 scores on the standardized quality rubric.
- Expert Approval: Subspecialty surgeons found zero clinical inaccuracies in the AI-generated forms, requiring no wording changes.
Limitations and Future Work
The authors acknowledge several limitations:
- Sample Size: The study only analyzed 15 institutional forms and 5 specific procedures.
- The "Human" Element: The study focuses on the written document, but informed consent is also a verbal process. The paper does not account for how these forms might change the actual conversation between doctor and patient.
- Patient Behavior: Even with simpler forms, some patients may still choose not to read them. Future research is needed to see if improved readability actually increases the rate at which patients engage with the documents.
- Future Directions: The authors suggest this framework could be extended to other materials, such as hospital promotional content, research consents, and real-time patient communication within Electronic Health Records (EHRs).
Summary updated