Interactive Clinical Reasoning and Sequential Diagnostic Inquiry

55 concepts48 papers

Large Language Models (LLMs) routinely achieve passing scores on standardized medical licensing examinations. However, when deployed at the bedside, these same models frequently struggle. This discrepancy highlights a fundamental gap between static knowledge retrieval and the dynamic, interactive reality of clinical practice.

To bridge this gap, a major research direction has emerged that reframes medical AI evaluation and architecture. Instead of treating diagnosis as a one-shot classification task where all information is known upfront, this paradigm models clinical reasoning as a sequential decision-making process driven by interactive inquiry.

The Evaluation Illusion of Static Benchmarks

Traditional medical AI benchmarks rely on static clinical vignettes—paragraphs of text that cleanly summarize a patient's history, physical exam, and lab results. The model's job is simply to output the correct diagnostic label.

Freya Gulamali characterizes this phenomenon as an "evaluation illusion" in [The evaluation illusion of large language models in medicine]. Static multiple-choice questions or single-turn prompts pre-package all necessary context, bypassing the most challenging phase of real-world medicine: information elicitation. In a live clinic, patients present with vague, unorganized, or ambiguous symptoms. The clinician must actively interview the patient, clarify contradictions, and decide which questions to ask or tests to order.

When models are forced to transition from passive summaries to active information gathering, their performance often degrades. As shown by Friederike Jungmann et al. in [Evaluation and mitigation of the limitations of large language models in clinical decision-making], models evaluated on raw patient cases struggle to autonomously gather information or systematically adhere to diagnostic guidelines. Furthermore, Nikita Mehandru et al. note in [Evaluating large language models as agents in the clinic] that static tests fail to account for how errors can cascade across a sequence of clinical actions, meaning high exam scores do not guarantee real-world safety.

Diagnosis as a Sequential Decision Process

To mimic human clinical intuition, automated systems must treat diagnosis not as a destination, but as a continuous trajectory of meaning-making. This concept aligns with the perspective of Kevin W. Eva et al. in [What’s in a Label? Is Diagnosis the Start or the End of Clinical Reasoning?], who argue that overemphasizing static diagnostic labels encourages premature cognitive closure.

Mathematically and algorithmically, this is achieved by framing clinical inquiry as a sequential decision process. Early work by Yuan Ling et al. in [Learning to Diagnose: Assimilating Clinical Narratives using Deep Reinforcement Learning] demonstrated that modeling clinical narratives incrementally allows systems to better handle noise and weigh evidence step-by-step.

[Initial Vague Symptom] 
          │
          ▼
┌───────────────────────────┐
│   Formulate Hypotheses    │◀─┐
└───────────────────────────┘  │
          │                    │ Iterative
          ▼                    │ Feedback
┌───────────────────────────┐  │ Loop
│ Select Next Inquiry/Test  │  │
└───────────────────────────┘  │
          │                    │
          ▼                    │
┌───────────────────────────┐  │
│  Observe New Information  │──┘
└───────────────────────────┘
          │
          ▼
  [Differential Diagnosis]

By utilizing Reinforcement Learning (RL), an agent can be trained to balance the value of new information against the "cost" (such as time, money, or patient discomfort) of obtaining it. For example, Victor Ardulov et al. applied Q-learning in [Robust diagnostic classification via Q-learning] to build a diagnostic system that actively requests the most informative symptoms while penalizing excessive data collection, making the system highly robust to missing data.

In modern LLM architectures, this sequential logic is enforced through multi-agent frameworks or specialized prompting strategies:

Simulating the Bedside: Interactive Benchmarks

If clinical AI must act as an autonomous agent, evaluation frameworks must transition from static datasets to interactive simulations. Researchers are increasingly using task-oriented dialogue systems to act as "Standardized Patients" that reveal symptoms only when prompted correctly.

Ruoyu Liu et al. introduced MMD-Eval in [Interactive Evaluation for Medical LLMs via Task-oriented Dialogue System], providing a hallucination-free environment to measure an LLM's proactivity in gathering information during multi-turn consultations. Similarly, [Simulating Viva Voce Examinations to Evaluate Clinical Reasoning in Large Language Models] by Christopher Chiu et al. leverages the structure of oral medical exams (viva voce) to force models to navigate diagnostic trajectories from ambiguous starting points, exposing human-like cognitive flaws such as anchoring bias.

This shift toward interactive evaluation also allows researchers to measure how well models handle real-world clinical messiness, such as demographic biases and systemic ambiguity. Max Lamparth et al. developed MENTAT [Moving Beyond Medical Exams: A Clinician-Annotated Fairness Dataset of Real-World Tasks and Ambiguity in Mental Healthcare], a psychiatric dataset designed to test language models within multi-turn interactions where patient data evolves dynamically.

By evaluating the entire interactive trajectory rather than just the final answer, these frameworks ensure that medical AI systems are safe, thorough, and capable of managing the open-ended uncertainties of actual clinical practice.

Key Foundational Work

Dynamic Network Updating Techniques for Diagnostic Reasoning (1991) — contributed 1 concept elements to this direction.
What’s in a Label? Is Diagnosis the Start or the End of Clinical Reasoning? (2016) — contributed 1 concept elements to this direction.
Implementing Troubleshooting with Batch Repair (2016) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_static_evaluation_limitations_9bee2728,
  title = {static evaluation limitations},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/static-evaluation-limitations/rdr_9bee2728221b4836b566408760aa1f2a},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-21},
}

This exact version (version 1, frozen text):

@misc{lacuna_static_evaluation_limitations_9bee2728_v1,
  title = {static evaluation limitations},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/static-evaluation-limitations/rdr_9bee2728221b4836b566408760aa1f2a/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-21},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake