Abstract
Large Language Models (LLMs) show promise in medical dialogue, but "end-to-end" applications in structured medical follow-ups often result in logic errors, repetitive questioning, and excessive token consumption. We present a modular framework that decomposes the follow-up process into structured stages, including semantic clustering and rule-driven flow control. Our evaluation demonstrates that this approach reduces dialogue turns by approximately 47% and slashes token consumption by over 80% while maintaining high accuracy across complex clinical forms.
Introduction
Medical follow-up is a high-stakes task requiring adherence to complex forms with nested logic and conditional skip patterns. While LLMs excel at natural conversation, they struggle to autonomously manage these rigid structures. In standard end-to-end configurations, models frequently suffer from information loss, uncontrolled dialogue flow (e.g., infinite loops), and significant operational inefficiency. This study bridges the gap between LLM flexibility and clinical reliability by introducing a scaffolded, modular architecture.
Proposed Modular Framework
The proposed framework moves the "logic" of the follow-up out of the LLM's prompt and into a structured program. The system is divided into five key components:
- Form Structure Construction: Paper-based clinical forms are converted into structured JSON with explicit logic.
- Question Preprocessing: An LLM semantically clusters similar questions to streamline the interview.
- Question Generation: Empathetic, human-like queries are generated based on these clusters.
- RAG-based Intent Extraction: Retrieval-Augmented Generation (RAG) maps patient responses back to structured data using few-shot examples.
- Dialogue Flow Control: A rule-driven module manages transitions, ensuring no questions are missed or repeated.
Experimental Setup
The framework was tested against an end-to-end LLM control group using 32 real-world medical follow-up forms covering common cancers. Three benchmark forms of varying complexity were used:
- Form-1 (Simple): 10 single-choice questions.
- Form-2 (Medium): 45 single-choice questions.
- Form-3 (High Complexity): 53 questions with complex skip logic and multiple-choice/fill-in-the-blank types.
Evaluations were conducted using Qwen-plus, DeepSeek-chat, and GPT-4, interacting with "Virtual Patients" designed with diverse conversational traits.
Results and Discussion
The modular approach significantly outperformed the end-to-end baseline in efficiency, cost, and stability.
Efficiency and Cost Reduction
The modular framework reduced average dialogue turns by 46.73%. More strikingly, token consumption was 5 to 8 times lower than the control group. In complex scenarios, end-to-end models often spiraled into redundant loops, whereas the modular system maintained a tight, purposeful flow.
Accuracy and Stability
While all models performed well on simple forms, the accuracy of end-to-end models plummeted on complex forms (Form-3) due to logic failures. The modular system maintained stable accuracy by offloading bookkeeping to the rule-based controller. It also eliminated common failure modes like premature termination and infinite loops.
Conclusion
LLMs are not "plug-and-play" for tasks requiring strict logical adherence. By using LLMs for language processing and rule-based systems for flow control, we can create robust, efficient clinical tools. Future work will focus on reducing rule dependency through better generalization and incorporating multimodal inputs like voice and images to improve patient accessibility.
Summary updated