Comparison of preoperative education by artificial intelligence versus traditional physicians in perioperative management of urolithiasis surgery: a prospective single-blind randomized controlled trial conducted in China

Hui Zhang, Xianjing Wang, Hongyan Luo, Weinan Zeng, Xuwei Hong, Jiarong Feng, Guangming Lu, Yongquan Su, Wenting Tong, Yang XiaoFrontiers in Medicine 2025 · DOI

Abstract

AI-assisted education, particularly via ChatGPT, can effectively reduce preoperative anxiety and improve patient engagement, though not to the level of physician-led education. Ernie Bot showed modest benefits. Further refinement of AI may enhance its role as a supplemental educational tool.

Introduction

Preoperative anxiety affects between 60% and 92% of surgical patients, triggering physiological stress responses that can complicate anesthesia and delay recovery. While thorough preoperative education is known to mitigate this, real-world clinical constraints—namely limited physician time and high patient volume—often result in perfunctory consultations.

This study investigates whether Large Language Models (LLMs) can bridge this gap by providing personalized, repeatable, and empathetic information exchange necessary to lower anxiety levels in patients undergoing urolithiasis (kidney stone) surgery.

Methods

The researchers conducted a single-center, single-blind randomized controlled trial (RCT) in China with 75 adult patients. Participants were randomized 1:1:1 into three groups:

  1. ChatGPT Group: Used GPT-4o for a 30-minute Q&A session.
  2. Ernie Bot Group: Used Baidu’s Ernie Bot 3.5 for a 30-minute Q&A session.
  3. Traditional Physician Group: Received an additional 30-minute face-to-face session with an attending urologist.

The primary metric was the State–Trait Anxiety Inventory (STAI), measured at four intervals. Secondary metrics included patient satisfaction, question counts, and an expert-led evaluation of the AI’s response quality.

Table 1: Baseline demographic and clinical characteristics of the study participants across the three intervention groups.

Key Results and Insights

The study demonstrated that all three interventions significantly reduced anxiety ($p < 0.05$), with the physician group achieving the greatest reduction, followed closely by ChatGPT.

The Engagement Paradox

Patients in the ChatGPT group asked nearly twice as many questions as those in the other two groups (an average of 23 vs. ~10-11). This suggests that the "infinite patience" and high readability of certain LLMs may lower the psychological barrier to inquiry, fostering deeper patient engagement than even a face-to-face doctor visit.

Quality and Safety

ChatGPT excelled in empathy, readability, and detail. However, it was the only group to produce minor inaccuracies—three identified errors—whereas the Urologist and Ernie Bot groups had zero. Ernie Bot’s performance was hampered by its conservative nature; strict regulatory compliance resulted in less detailed, less empathetic responses, leading to lower patient satisfaction.

Table 2: Comparison of preoperative intervention results, including psychological metrics and AI performance evaluations.

Durability of Effect

At the 3h pre-op mark, the physician and ChatGPT groups maintained lower anxiety levels, while the Ernie Bot group showed a non-significant rise in anxiety. Satisfaction was high and comparable between the Physician and ChatGPT groups, while Ernie Bot scored significantly lower due to its brief, "canned" responses.

Limitations and Future Work

The authors acknowledge several constraints, including the homogeneity of the sample (only healthy ASA I patients) and the retrospective nature of the AI content review. Future implementations require real-time safety oversight to prevent hallucinations. Integrating these chatbots directly into Hospital Information Systems (HIS) could provide even more personalized, data-driven education in the future.

Summary updated

Abstract

AI-assisted education, particularly via ChatGPT, can effectively reduce preoperative anxiety and improve patient engagement, though not to the level of physician-led education. Ernie Bot showed modest benefits. Further refinement of AI may enhance its role as a supplemental educational tool.
BibTeX
@inproceedings{zhang2025comparison,
  author = {Hui Zhang and Xianjing Wang and Hongyan Luo and Weinan Zeng and Xuwei Hong and Jiarong Feng and Guangming Lu and Yongquan Su and Wenting Tong and Yang Xiao},
  title = {{Comparison of preoperative education by artificial intelligence versus traditional physicians in perioperative management of urolithiasis surgery: a prospective single-blind randomized controlled trial conducted in China}},
  booktitle = {Frontiers in Medicine},
  year = {2025},
  doi = {10.3389/fmed.2025.1543630},
}
Figure