A prospective randomized study of patient-centered AI for clinical visit preparation

Xincheng Tan, Matthew Thompson, Xiang Ji, Angelica Willis, Will Vaughan, Roma Ruparel, Michael A. Sanchez, Sean Chang, Joseph Xu, Bhavna Daryani, Preeti Singh, Pinal Bavishi, Kejia Chen, Fan Zhang, Priya Gupta, Ernest Lim, Christina Chen, Bob Lou, Dimitrios Antos, Zach Wasson, Ray Luo, Fadi Yousif, Florence Thng, Mark Malhotra, Joseph Breda, Daniel McDuff, Kavi Goel, Katherine Chou, Dale R. Webster, Avinatan Hassidim, Yossi Matias, Po-Hsuan Cameron Chen, Yun Liu, Mike Schaekermann, Quang Duong, Jackie M. BarrarXiv 2026 · arXiv

TL;DR

Researchers from Google Research conducted a large-scale randomized study demonstrating that a structured, LLM-powered "visit preparation agent" significantly improves patient readiness and confidence for medical appointments. By guiding users through a specific clinical workflow, the AI achieved higher clinician-rated utility and patient satisfaction than a standard chatbot, all while maintaining a high bar for medical safety.

What Problem Does This Paper Address?

The modern clinical encounter is under immense pressure. In the U.S., primary care visits often last only 15–20 minutes; in other countries, they can be as short as 10 minutes. Patients frequently arrive at these brief appointments feeling anxious, forgetting key symptoms, or unsure of what to ask, leading to suboptimal care.

While millions of adults are already turning to general-purpose AI chatbots for health advice, the clinical community remains skeptical: Does an open-ended conversation actually help a patient prepare? Or does it just provide a "hallucinated" sense of confidence? This paper investigates whether a Large Language Model (LLM) can be transformed into a reliable clinical tool by wrapping it in a structured, patient-centered workflow.

Methods

The team developed a visit preparation agent powered by Gemini 2.5 Flash. Unlike a standard chatbot that simply responds to prompts, this agent was designed with a structured 10-step workflow (see Figure 1) that includes:

  1. Context gathering: Identifying the visit type and current concerns.
  2. Expectation setting: Explaining what likely happens during the visit (e.g., tests, physical exams).
  3. Information provision: Explaining medical concepts in plain language.
  4. Agenda building: Helping the patient prioritize their top three questions.
  5. Summarization: Generating a "prep sheet" for the patient to take to their doctor.
Crossover analysis showing that the visit prep agent increased utility while maintaining high reliability.

The researchers conducted a prospective, three-arm randomized study with 802 U.S. adults. Participants were assigned to either a Control (minimal prompting), a Standard visit prep agent, or a Concise variant. The study utilized a "dual-perspective" evaluation: blinded board-certified clinicians reviewed the transcripts for safety and utility, while patients provided self-reports on their preparedness and satisfaction.

Datasets

The study did not rely on existing public benchmarks. Instead, it generated a novel dataset of 866 health-related conversations across a wide range of clinical domains, including cardiovascular, metabolic, and musculoskeletal conditions (see Figure 3 for participant demographics).

For the clinical evaluation, a random sample of 272 completed sessions was reviewed by independent physicians. Patient outcomes were captured via immediate post-interaction surveys (n=460) and a 14-day follow-up survey (n=95) to see if the preparation actually influenced their real-world doctor visits.

Key Insights and Ideas

  • Structure Trumps Raw Capability: Even though all study arms used the same underlying model (Gemini 2.5 Flash), the structured agent significantly outperformed the control. This suggests that for high-stakes domains like healthcare, the interaction design and system prompting are just as critical as the model's parameters.
  • The "Divergence" of Perspectives: One of the most striking findings was that clinician ratings of quality did not correlate with patient satisfaction. A patient might feel very helped by a conversation that a doctor finds clinically incomplete, and vice versa. This highlights the necessity of "dual-perspective" evaluations for medical AI.
  • Utility without Safety Trade-offs: A common fear is that making an AI more "helpful" (proactive, detailed) increases the risk of "unsafe helpfulness" (giving bad medical advice). This study showed that the structured agent increased clinical utility (comprehensiveness and alignment) without compromising the high baseline of medical safety (over 90% in all arms).
Crossover analysis showing that the visit prep agent increased utility while maintaining high reliability.

Key Results

The structured visit preparation agent showed clear statistical superiority over the control:

  • Clinician Perspective: The agent achieved a 16.6% higher rate of comprehensive preparation (67.2% vs. 50.6%) and was significantly better at aligning with the patient's specific health context (98.9% vs. 90.4%).
  • Patient Perspective: Users reported 16.8% higher visit preparedness and 14.3% higher confidence in knowing what to ask their doctor. Overall satisfaction was also significantly higher (74.0% vs. 62.0%).
  • Safety: Medical safety and appropriateness remained high (>93%) across all study arms, with no significant differences, suggesting the structured flow is a low-risk way to empower patients.
Distribution of patient ratings showing significant gains in preparedness and confidence.

Limitations and Future Work

The authors acknowledge several hurdles for real-world deployment. First, there was a significant attrition rate, with roughly 42% of conversations ending before the full 10-step flow was completed. This suggests the need for even more streamlined or adaptive interfaces.

Second, while patients felt more prepared, the study had limited power to prove that this translated into better clinical outcomes (like improved diagnostic accuracy or treatment adherence) during the actual doctor visit. Future research should involve the clinicians who treat these prepared patients to see if the "prep sheets" actually improve the efficiency and quality of the face-to-face encounter. Finally, the study cohort was limited to Android users of the Google Health app, who may be more tech-literate than the general population, necessitating further studies on broader, more diverse demographic groups.

Summary updated

Abstract

Patients often arrive at brief, time-constrained clinical visits unsure of what to expect or ask, and millions of adults now turn to general-purpose AI chatbots to prepare beforehand. Yet whether conversational AI improves visit readiness remains unclear. We designed a large language model (LLM)-based visit preparation agent to guide patients before an appointment by gathering clinical context, setting visit expectations, providing relevant health information, building a prioritized list of questions for their clinician, and generating a summary. In a prospective consented randomized study in the Google Health app, 802 U.S. adults held 866 health-related conversations with the visit preparation agent (standard or concise variant) or a minimally prompted control chatbot built on the same LLM. The visit preparation agent significantly outperformed the control across blinded clinician evaluations and patient self-reports. In clinician reviews of completed sessions (n = 272), the agent achieved a 16.6% higher rate of comprehensive preparation (67.2% vs. 50.6%; P < 0.05) and an 8.5% higher rate of alignment with patients' health context (98.9% vs. 90.4%; P < 0.01), while maintaining high medical safety and appropriateness across all study arms. Patient post-surveys (n = 460) showed 16.8% higher visit preparedness (73.8% vs. 57.0%; P < 0.01), 14.3% higher confidence in knowing what to ask (81.4% vs. 67.1%; P < 0.01), and 12.0% higher overall satisfaction (74.0% vs. 62.0%; P < 0.05). Clinician and participant metrics were not significantly correlated, indicating that expert review and patient self-assessment capture complementary dimensions of patient-facing AI. These findings suggest that LLM chatbots can safely help patients prepare for clinical visits, with greater benefit when following a structured workflow, and motivate future studies on in-visit communication and clinical outcomes.
BibTeX
@misc{tan2026prospective,
  author = {Xincheng Tan and Matthew Thompson and Xiang Ji and Angelica Willis and Will Vaughan and Roma Ruparel and Michael A. Sanchez and Sean Chang and Joseph Xu and Bhavna Daryani and Preeti Singh and Pinal Bavishi and Kejia Chen and Fan Zhang and Priya Gupta and Ernest Lim and Christina Chen and Bob Lou and Dimitrios Antos and Zach Wasson and Ray Luo and Fadi Yousif and Florence Thng and Mark Malhotra and Joseph Breda and Daniel McDuff and Kavi Goel and Katherine Chou and Dale R. Webster and Avinatan Hassidim and Yossi Matias and Po-Hsuan Cameron Chen and Yun Liu and Mike Schaekermann and Quang Duong and Jackie M. Barr},
  title = {{A prospective randomized study of patient-centered AI for clinical visit preparation}},
  year = {2026},
  eprint = {2610.41437},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.41437},
}
Figure