TL;DR
When researchers evaluate medical AI using APIs, they aren't seeing the same outputs that patients see on consumer chatbot interfaces. This study reveals that systematic differences in formatting, conversational behavior, and clinical content exist between API and chatbot access modes—even when using the exact same underlying model version—creating a critical blind spot for AI safety auditing.
What Problem Does This Paper Address?
To evaluate the safety and accuracy of Large Language Models (LLMs) in healthcare, researchers typically query the model's Application Programming Interface (API) because it allows for automated, high-throughput testing. However, everyday consumers interact with these models through web-based chatbot interfaces (like ChatGPT or the specialized ChatGPT Health).
The authors tackle a fundamental question: Are API-based evaluations a valid proxy for actual consumer experiences? Chatbot interfaces often include undocumented system instructions, hidden guardrails, and unique orchestration layers that aren't present in a raw API call. If the outputs diverge significantly, current AI auditing practices may be mismeasuring the risks and utility of the tools patients actually use.
Methods
The researchers designed a tightly controlled experiment to isolate the effect of the "access mode." They compared responses to identical health queries across two model generations (GPT-5.3 and GPT-5.4) using three access modes:
- GPT API: Raw programmatic access with an empty system prompt and matched parameters.
- Standard ChatGPT: The standard consumer web interface.
- ChatGPT Health: A dedicated, health-specific consumer experience that automatically triggers when personal health questions are detected.
To ensure a fair comparison, they used fresh sessions with no conversation history, memory, or user customizations. Each question was submitted three times per mode to account for run-to-run randomness. Responses were evaluated across four dimensions: response form (length, readability), behavior (disclaimers, follow-up questions), referenced web sources, and clinical content (extracted using a clinician-reviewed pipeline).
Datasets
The study utilized HealthCareMagic-100k, a dataset containing real, patient-authored medical questions from an online consultation platform. The authors sampled 50 questions, filtering out queries that explicitly addressed the recipient as a human doctor (e.g., "Doctor, what should I do?"), ensuring the prompts mirrored natural queries a user would type into an AI chatbot.
Key Insights and Ideas
- Access Mode is a Hidden Variable: Holding the model version constant is not enough. The wrapper around the model (the interface) fundamentally alters the behavior, formatting, and clinical focus of the response.
- The "Flip-Flop" Effect: Differences between access modes are not stable across model updates. For example, in GPT-5.3, the chatbot interface produced text that was easier to read and more heavily formatted than the API. In GPT-5.4, this trend completely reversed—the interface responses became harder to read and less formatted than their API counterparts.
- Search Hallucination/Divergence: When using web-retrieval features (GPT-5.4), the overlap of cited URLs between the API and the interface dropped significantly, meaning the two modes are effectively looking at different parts of the web to answer the same medical question.
Key Results
The quantitative analysis highlighted stark contrasts across the access modes:
- Response Length & Formatting (GPT-5.3): Standard ChatGPT responses were significantly longer (averaging 377 words) than API responses (268 words) and contained far more section headings, lists, and emojis.
- Engagement Behavior: In GPT-5.3, consumer interfaces invited further interaction much more aggressively, asking follow-up questions in up to 91.3% of runs (for ChatGPT Health) compared to just 54.0% for the API.
- Source Overlap (GPT-5.4): While the API and interface cited a similar number of sources, the actual web pages chosen differed wildly. The Jaccard overlap for specific web pages across access modes was a meager 10.9%, compared to a 17-18% overlap within the same mode.
Limitations and Future Work
The study evaluated access modes under ideal, highly controlled conditions (single-turn interactions, no user data connected). In the real world, consumer chatbots leverage persistent memory, custom user instructions, and integrations with health apps or Electronic Health Records (EHRs). The authors note that these features will likely widen the gap between API evaluations and consumer experiences even further.
The paper concludes with an urgent call to action for regulators and AI providers: model developers must provide specialized "auditing APIs" that faithfully replicate the exact system prompts, tools, and guardrails experienced by everyday patients.
Summary updated