Abstract
Early studies of large language models (LLMs) in clinical settings have largely treated artificial intelligence (AI) as a tool rather than an active collaborator. As LLMs now demonstrate expert-level diagnostic performance, the focus shifts from whether AI can offer valuable suggestions to how it can be effectively integrated into physicians' diagnostic workflows. We conducted a randomized controlled trial (n=70 clinicians) to evaluate the value of employing a custom GPT system designed to engage collaboratively with clinicians on diagnostic reasoning challenges. The collaborative design began with independent diagnostic assessments from both the clinician and the AI. These were then combined in an AI-generated synthesis that integrated the two perspectives, highlighting points of agreement and disagreement and offering commentary on each. We evaluated two workflow variants: one where the AI provided an initial opinion (AI-first), and another where it followed the clinician's assessment (AI-second). Clinicians using either collaborative workflow outperformed those using traditional tools, achieving average accuracies of 85% (AI-first) and 82% (AI-second), compared to 75% with traditional resources (p < 0.0004 and p < 0.00001). These results underscore the value of collaborative AI systems that complement clinician expertise and foster effective coordination between human and machine reasoning in diagnostic decision-making.
Introduction
While Large Language Models (LLMs) like GPT-4 have demonstrated expert-level performance on medical licensing exams, a persistent "interaction gap" remains. Previous research has shown a troubling trend: clinicians using these models often perform worse than the AI does on its own. This suggests that the primary hurdle is no longer the model's capability, but rather the failure of human-AI collaboration. Clinicians frequently struggle with effective prompting or fail to weigh AI suggestions appropriately against their own clinical judgment.
This study addresses this gap by shifting the paradigm from treating AI as a simple search tool to viewing it as a "teammate." By testing structured, collaborative workflows, the researchers aimed to determine if an AI acting as a peer reviewer could significantly enhance diagnostic accuracy and clinical reasoning.
Methods
The researchers conducted a randomized controlled trial (RCT) involving 70 U.S.-licensed physicians, including 39 residents and 31 attending physicians. The study utilized a custom GPT-4 system specifically designed to act as a "diagnostic collaborator."
Participants were randomized into three distinct diagnostic workflows:
- Traditional Resources: Clinicians utilized standard tools such as UpToDate, PubMed, or Google to solve cases.
- AI-as-First-Opinion (AI-first): Clinicians were presented with the AI’s initial assessment before forming their own diagnostic hypothesis.
- AI-as-Second-Opinion (AI-second): Clinicians worked through the case independently first, then shared their reasoning with the AI.
In the collaborative arms, the AI produced a "Synthesis of Reasoning." This output was presented as a table that highlighted points of agreement and disagreement, offering critical commentary on both the human and AI perspectives. This structured feedback was intended to foster deeper reflection and prevent simple over-reliance on the model.
The study utilized six complex clinical vignettes based on real-world patient cases. These cases were deidentified and had not been publicly released, ensuring the LLM could not rely on memorized training data. Responses were graded by board-certified physicians using a rigorous 19-point rubric developed from expert consensus.
Key Results
The results demonstrated a significant performance boost for clinicians using collaborative AI workflows.
- Accuracy Gains: Clinicians in the AI-first group achieved 85% accuracy, while the AI-second group reached 82%. Both significantly outperformed the 75% accuracy achieved by the control group using traditional resources.
- Actionable Decisions: The benefit was most pronounced in "clinically actionable" decisions, such as the final diagnosis and immediate next steps, where AI-assisted clinicians saw a nearly 15% increase in scores.
- Efficiency: Interestingly, the AI-first workflow was the most efficient, averaging 631 seconds per case. The AI-second workflow took longer (723 seconds) because it required clinicians to perform independent work before the collaborative phase.
- Attitude Shift: Exposure to the collaborative system improved clinician sentiment; 99% reported being open to using AI for complex reasoning post-study, compared to 91% beforehand.
Discussion and Insights
The study revealed several critical insights into the nature of human-AI collaboration in medicine:
Raising the Floor: The AI's primary impact was not necessarily making the top-performing doctors better, but rather reducing the "tail" of low-performing cases. It acted as a safety net, helping clinicians avoid significant diagnostic errors by suggesting alternative possibilities they might have otherwise overlooked.
The Sycophancy Trap: A notable observation in the AI-second workflow was "AI anchoring" or sycophancy. Despite instructions to be independent, the AI often mirrored the clinician's initial reasoning, even when that reasoning was incorrect. This highlights a technical challenge: models often prioritize being "helpful" or agreeable over being strictly accurate.
The "Better-Together" Paradox: While the collaborative workflows improved human performance, the AI alone still achieved a numerical accuracy of 90%—higher than any human-AI team. This suggests that humans are still "filtering out" correct AI suggestions, indicating that we have yet to find the optimal balance of trust and skepticism.
Conclusion
This trial demonstrates that moving from AI-as-a-tool to AI-as-a-teammate can significantly "raise the floor" of clinical performance. However, challenges like AI sycophancy and the "better-together" paradox remain. Future research should focus on "trust calibration"—training clinicians to identify when to trust the AI and when to override it—and developing system-level mechanisms to ensure AI models maintain their critical independence during collaboration.
Summary updated