Crucial Role of Understanding in Human-Artificial Intelligence Interaction for Successful Clinical Adoption

Seong Ho Park, Curtis P. LanglotzKorean 2025 · DOI · OpenAlex

Introduction

Despite the U.S. Food and Drug Administration (FDA) approving over 1,000 AI-enabled medical devices, the transition from laboratory success to real-world clinical utility remains slow. This "implementation gap" suggests that algorithmic accuracy alone is insufficient for clinical success. This paper argues that the missing link is a robust understanding of Human-AI Interaction (HAII). To achieve successful integration, the medical community must move beyond model performance to address behavioral responses like automation bias, manage the unique challenges of Large Language Models (LLMs), and mitigate the digital workload that AI can inadvertently impose on clinicians.

The Human-AI Interaction Bottleneck

The primary challenge in medical AI implementation is the complex and often unpredictable dynamic between healthcare professionals and digital tools. Clinicians frequently oscillate between two extremes: automation bias, where they over-rely on AI suggestions (even incorrect ones), and outright dismissal, where they ignore valid AI insights due to a lack of trust. The "black box" nature of many models exacerbates this, as practitioners often lack the necessary context to determine when a model’s output is reliable. Successful synergy requires humans and AI to complement each other's specific weaknesses rather than compounding them.

Methodology and Framework

This editorial synthesizes high-impact research to establish a framework for successful HAII, categorizing challenges into three core pillars:

  1. Behavioral Responses: Evaluating how clinical experience levels influence trust and susceptibility to automation bias.
  2. Large Language Models (LLMs): Assessing unique risks such as "hallucinations" and "sycophancy" in clinical diagnostic settings.
  3. Digital Workload: Investigating the impact of AI tools on clinician burnout and cognitive fatigue.

Evidence was drawn from diverse sources, including a nationwide survey of 6,726 radiologists in China, diagnostic studies involving GPT-4's reasoning, and performance evaluations of physicians using generative AI for clinical documentation.

Key Insights

The Experience Paradox and the Explainability Gap

Research indicates that while AI can significantly boost the performance of less-experienced practitioners (e.g., residents), these individuals are also the most vulnerable to automation bias. Conversely, experts often see minimal gains. Furthermore, current "Explainable AI" (XAI) tools, such as saliency maps, often fail to bridge the trust gap. In one cited study, even when provided with XAI tools, practitioner accuracy remained 9.1% below the baseline when faced with a biased AI model, suggesting that current interpretability methods are insufficient for critical decision-making.

The LLM Performance Dip

A striking finding in recent diagnostic studies is that physicians using GPT-4 for diagnosis performed significantly worse (76% accuracy) than the LLM did in a standalone capacity (92%). This performance degradation suggests that without specific training in prompt engineering and a deep understanding of LLM behavior, the "human-in-the-loop" can actually hinder the system's overall effectiveness.

The Workload Myth

While AI is often marketed as a solution to clinician burnout, data suggests a more complex reality. A comprehensive survey in China found a direct correlation between the frequency of AI use for radiology interpretation and increased work-related emotional exhaustion. While AI may streamline simple tasks, it can increase the cognitive load and time required for complex, abnormal cases, leading to "digital fatigue."

Key Results Summary

  • Accuracy Decline: Practitioner accuracy dropped by 11.3% when interacting with biased AI models.
  • Performance Gap: GPT-4’s standalone diagnostic accuracy (92%) significantly outperformed physician-assisted accuracy (76%) in the absence of specialized training.
  • Burnout Correlation: Increased AI usage in radiology is linked to higher levels of emotional exhaustion among practitioners, refuting the idea that AI is a universal "burnout killer."

Future Directions

To unlock the full potential of medical AI, the field must move beyond simple heatmaps toward uncertainty quantification, where models explicitly communicate their confidence levels. Additionally, medical curricula must evolve to include AI literacy and prompt engineering as fundamental clinical skills. Finally, AI deployment must be precisely tailored to specific clinical workflows to ensure it reduces, rather than increases, the cognitive burden on healthcare providers. Successful clinical adoption depends on treating Human-AI Interaction as a rigorous science rather than an implementation afterthought.

Summary updated

BibTeX
@inproceedings{park2025crucial,
  author = {Seong Ho Park and Curtis P. Langlotz},
  title = {{Crucial Role of Understanding in Human-Artificial Intelligence Interaction for Successful Clinical Adoption}},
  booktitle = {Korean},
  year = {2025},
  doi = {10.3348/kjr.2025.0071},
}
Figure