Triplet-Structured Knowledge Integration for Multi-Turn Medical Reasoning

Introduction

Large Language Models (LLMs) have demonstrated remarkable proficiency in static medical examinations. However, real-world clinical practice is inherently conversational and iterative. In these multi-turn interactions, a diagnosis is constructed incrementally as patients reveal symptoms, travel history, and medication details over time.

Standard LLMs often struggle in these scenarios because they treat dialogue as a "flat sequence" of text. As conversations lengthen, critical clinical facts mentioned early on can be obscured by subsequent dialogue, leading to "premature closure"—where the model reaches a diagnostic conclusion before gathering all necessary information—or a failure to synthesize disparate facts.

Expand

Source, p. 2

The TriMediQ Framework

To address these limitations, we introduce TriMediQ, a framework that shifts medical reasoning from raw text to structured data. TriMediQ dynamically constructs a patient-specific Knowledge Graph (KG) during the interview and injects these structured relationships back into the LLM.

The system comprises four primary components:

1. Patient System: A frozen LLM that simulates a patient based on hidden medical records.
2. Triplet Generator: A frozen LLM that extracts "triplets" (Subject-Relation-Object) from patient responses, such as [Patient, HAS_SYMPTOM, Fatigue], using constrained prompting to align with medical ontologies like UMLS.
3. Knowledge Graph (KG): An evolving graph that explicitly links clinical entities across multiple turns, preserving the context of the entire interaction.
4. Projection Module: The core innovation and only trainable component. It utilizes a Graph Convolutional Network (GCN) to encode the KG's structure into a vector, which is then converted into a "prefix embedding" (soft prompt) for the Expert LLM.

Expand

Source, p. 5

Experimental Evaluation

TriMediQ was evaluated on two interactive benchmarks: iMedQA, a multi-turn reformulation of the MedQA dataset, and iCRAFT-MD, a dermatology dataset featuring 140 complex cases. The researchers tested the framework across several state-of-the-art LLMs, including Llama-3, Qwen-2.5, and DeepSeek.

Key Insights and Results

The integration of structured knowledge yielded significant improvements in diagnostic performance:

• Accuracy Gains: TriMediQ achieved a 10.4% absolute improvement in accuracy on iMedQA using Llama-3-70B. Smaller models also benefited, with Llama-3-8B seeing an 8.2% gain on iCRAFT-MD.
• Relational Awareness: Comparisons between simple fact pooling and graph-based projection confirmed that the topology of clinical data—how facts relate—is crucial for effective multi-hop reasoning.
• Resilience to Dialogue Fatigue: While standard models often plateau after approximately six turns, TriMediQ's accuracy continues to scale with dialogue depth (up to 10 turns), demonstrating superior long-horizon reasoning.
• Efficiency: By keeping the large "Expert" LLM frozen and only training the lightweight projection module, TriMediQ offers a computationally efficient alternative to full model fine-tuning.

Conclusion

While promising, the current framework is constrained by its reliance on a specific triplet schema. Future iterations could incorporate richer ontologies to capture more nuanced medical relationships. Additionally, the authors plan to evaluate TriMediQ against more naturalistic clinician-patient interactions and expand its utility toward conversational decision support, such as prioritizing diagnostic tests or treatments.

@misc{meng2025triplet,
  author = {Zhaohan Meng and Zaiqiao Meng and Siwei Liu and Iadh Ounis},
  title = {{Triplet-Structured Knowledge Integration for Multi-Turn Medical Reasoning}},
  year = {2025},
  eprint = {2510.03536},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2510.03536},
}