Introduction
In the contemporary healthcare landscape, patients frequently access their medical data—such as radiology reports and CT scans—through online portals long before they can consult with a physician. This delay creates a significant "information gap," often resulting in patient anxiety and a reliance on unverified online sources. While Large Language Models (LLMs) like GPT-4o and Claude 3.5 Sonnet demonstrate impressive performance on medical licensing exams, their ability to support a layperson's "sense-making" process remains largely untested. This research moves beyond academic benchmarks to perform a task-centric evaluation of Generative AI (GenAI) in supporting patient informational needs.
Methodology
The study employed a multi-stage approach to bridge the gap between AI capability and patient utility.
1. Formative Need-Finding
The researchers conducted simulated consultations where caregivers discussed a fictitious lung CT scan with a professional radiologist. Through inductive thematic analysis, they identified ten core informational needs, including clarifying medical terminology, understanding the implications of findings, and locating specific issues on the scan.
2. Comparative Evaluation and Dataset
Based on these needs, a high-quality multimodal dataset was developed. It consists of 25 visual question-answering (VQA) problems, each pairing a real-world patient question with two lung CT slices and a corresponding radiology report.
The performance of GPT-4o and Claude 3.5 Sonnet was evaluated in both "native" and "persona-prompted" modes. Responses were assessed for correctness by experts and analyzed for relevance and informativeness using Gricean maxims.
Key Findings: The Multimodality Gap
The evaluation revealed a stark contrast between the models' general medical knowledge and their practical utility in a clinical context.
The "Textbook Trap" and Verbosity
AI models frequently fell into a "textbook trap," providing generic information about diseases or statistics rather than tailoring answers to the specific patient's report. Furthermore, the models exhibited high verbosity; Claude 3.5 Sonnet, for instance, produced nearly double the word count of a human expert. While prompting for succinctness reduced length, it often led to the omission of clinically vital information.
Failure in Visual Grounding
Despite their multimodal architecture, both models struggled significantly with "Locating Issues." While a radiologist can intuitively point to "bubbly spots" or "white dots" on a scan, the AI models often provided generic instructions on how to read a scan or incorrectly claimed an inability to localize findings.
Error Rates and Metric Limitations
The study found alarmingly high error rates: 20% for GPT-4o and 40% for Claude 3.5 Sonnet. These errors included hallucinations and failures in causal reasoning. Crucially, the researchers noted that standard AI metrics, such as Universal Sentence Encoder (USE) similarity scores, failed to capture these clinical inaccuracies, often giving high scores to fundamentally incorrect answers.
Discussion: From Automation to Augmentation
The results suggest that current GenAI models are not yet ready for autonomous patient support. The authors argue for a philosophical shift from "AI-as-automation" (replacing the doctor) to "AI-as-augmentation." In this paradigm, AI is designed to prepare patients for their upcoming medical appointments, helping them formulate better questions and understand basic terminology without attempting to provide a definitive diagnosis or localize complex visual findings.
Conclusion
While GenAI holds promise for democratizing medical information, significant hurdles remain regarding accuracy and multimodal integration. Future work will focus on developing more robust measurement scales and exploring techniques like Retrieval-Augmented Generation (RAG) to mitigate the high error rates observed in "out-of-the-box" models.
Summary updated