Abstract
Medical dialogue generation (MDG) has gained increasing attention due to its substantial practical value. Previous works typically employ a sequence-to-sequence framework to generate medical responses by modeling dialogue context as sequential text with annotated medical entities. While these methods have been successful in generating fluent responses, they fail to provide process explanations of reasoning and require extensive entity annotation. To address these limitations, we propose the method Bootstrap Prompting for Explicit Reasoning in MDG (BP4ER), which explicitly models MDG's multi-step reasoning process and iteratively enhances this reasoning process. We employ a least-to-most prompting strategy to guide a large language model (LLM) in explicit reasoning, breaking down MDG into simpler sub-questions. These sub-questions build on answers from previous ones. Additionally, we also introduce two distinct bootstrapping techniques for prompting, which autonomously correct errors and facilitate the LLM's explicit reasoning. This approach eliminates the need for entity annotation and increases the transparency of the MDG process by explicitly generating the intermediate reasoning chain. The experimental findings on the two public datasets indicate that BP4ER outperforms state-of-the-art methods in terms of both objective and subjective evaluation metrics.
Introduction
Medical Dialogue Generation (MDG) aims to build automated systems capable of interacting with patients to provide preliminary diagnoses or medical advice. Traditionally, this task has been treated as a standard sequence-to-sequence (Seq2Seq) problem: a model takes a dialogue history as input and generates a response as output.
However, this "black box" approach faces two critical challenges. First, it lacks interpretability; patients and clinicians cannot see the underlying logic behind a model's recommendation. Second, it suffers from an annotation tax, as high-performing models usually require thousands of manually labeled medical entities (symptoms, diseases, and tests) to achieve accuracy. BP4ER (Bootstrap Prompting for Explicit Reasoning) addresses these issues by transforming MDG into a transparent, multi-step reasoning process that requires no manual entity annotation.
The BP4ER Framework
BP4ER moves away from direct text-to-text generation by decomposing the physician's thought process into a structured reasoning chain. This is achieved through three primary components:
1. The Multi-Step Reasoning Chain
Instead of generating a response in one go, BP4ER prompts the model to answer three sequential sub-questions:
- S1 (Patient State Tracking): What is the patient's current condition based on the history?
- S2 (Next Diagnosis Decision): What is the physician's next logical diagnostic step or decision?
- S3 (Medical Response): What is the final response to the patient?
2. Least-to-Most Prompting (LMP)
To ensure logical consistency, the framework utilizes a "least-to-most" strategy. The model generates the answer to S1 first, then uses that answer to inform S2, and finally uses the combined context of S1 and S2 to generate S3. This hierarchical approach ensures that the final response is grounded in a documented assessment of the patient's state.
3. Iterative Bootstrapping
To refine the model's reasoning without human intervention, BP4ER introduces two self-correction techniques:
- AP-Bootstrap (Answer-Providing): When a model generates an incorrect response, the system provides the "gold" (correct) answer and prompts the model to "reason backward." This helps the model discover the logical path required to reach the correct conclusion.
- PR-Bootstrap (Prompt-Revising): The model explores multiple reasoning paths by slightly varying its few-shot examples. By identifying consistent reasoning chains that lead to the same correct answer, the system autonomously curates high-quality training data.
Experimental Results
The researchers evaluated BP4ER on two benchmark datasets: MedDG (17K dialogues on gastrointestinal diseases) and KaMed (63K dialogues covering 300+ diseases).
Quantitative Performance
BP4ER demonstrated significant improvements over state-of-the-art (SOTA) models like DFMed and MedPIR. On the MedDG dataset, the framework achieved a relative increase of 23.42% in ROUGE-1 and 11.94% in ROUGE-2 scores compared to the base ChatGLM-6B model. Furthermore, the model showed higher DISTINCT-2 scores, indicating that the generated responses were more diverse and less prone to repetitive phrasing.
Human Evaluation
In blind tests conducted by professional medical annotators, BP4ER consistently outscored baseline models across three key dimensions:
- Fluency: 4.00/5
- Coherence: 3.50/5
- Correctness: 3.52/5
While a gap still exists between AI and "Gold" human responses, BP4ER significantly narrowed this margin compared to previous automated methods.
Key Insights and Discussion
Reasoning as a Substitute for Labeling
A primary finding of this research is that explicit reasoning chains can effectively replace expensive entity annotations. By forcing the model to articulate the patient's state (S1), it naturally identifies relevant symptoms and medical logic without needing explicit labels for every medical term in the training set.
The Value of Interpretability
Often, adding layers of explainability to an AI model results in a performance penalty. However, in the case of BP4ER, the explicit reasoning process actually improved the quality of the final response. The reasoning chain acts as a constraint that keeps the model grounded in the dialogue context, reducing the likelihood of irrelevant or generic replies.
Limitations and Future Work
Despite its success, BP4ER has higher computational costs because it generates more tokens (the reasoning chain) for every response. Additionally, like all Large Language Models, it is susceptible to hallucinations if the patient's description is highly ambiguous. Future research will focus on integrating external medical knowledge bases to provide the model with deeper specialized knowledge, such as specific drug dosages and rare disease contraindications.
Summary updated