Can LLMs Support Medical Knowledge Imputation? An Evaluation-Based Perspective

Xinyu Yao, Aditya Sannabhadti, Holly M. Wiberg, Karmel S. Shehadeh, Rema PadmanarXiv 2025 · DOI · arXiv

Introduction

Medical Knowledge Graphs (KGs) are essential for clinical decision support, yet they often suffer from incompleteness. As medical coding systems like ICD and ATC evolve, many valid drug-disease relationships remain unmapped in structured databases. While Large Language Models (LLMs) are frequently proposed as a solution for "knowledge imputation"—filling these gaps using their vast training data—this study evaluates their reliability in high-stakes clinical contexts.

Methodology

The researchers evaluated four state-of-the-art models: GPT-4o, OpenAI o3-mini, Perplexity-R1, and Perplexity-Sonar. A "ground truth" benchmark was established by integrating gold-standard databases, including UMLS, DrugBank, and PrimeKG, resulting in over 100,000 known drug-disease relationships.

The evaluation utilized three distinct prompt strategies:

  • Prompt A: A simple list request.
  • Prompt B: A granular request for Level 5 ATC codes.
  • Prompt C: A highly structured JSON format including disease descriptions.

Performance was measured using Success Rate (the ability to identify at least one correct treatment) and Recall (the percentage of all known treatments identified).

Key Results: The Precision-Recall Paradox

The study reveals a significant "precision-recall paradox." As prompts become more structured and detailed (Prompt C), models become highly successful at identifying at least one correct drug. However, this precision comes at the expense of comprehensive coverage.

Comparison of Success Rate and Recall across different prompt styles and models.

As shown in the figure above, while Success Rates for Prompt C reached as high as 0.9667 for models like GPT-4o and Perplexity-Sonar, the corresponding Recall scores were dismal, ranging from 0.0074 to 0.0149. Even under the most permissive conditions (Prompt A), the highest recall achieved was only 0.0842. This indicates that while LLMs can reliably name a drug for a condition, they fail to map the full therapeutic landscape.

Discussion

The "Echo Chamber" Effect

A concerning finding was that more detailed prompts (Prompt C) led to higher agreement between different models (Dice scores up to 0.85) while simultaneously decreasing their alignment with the actual medical KG (below 0.03). This suggests that LLMs may share common biases or "reasoning shortcuts" that diverge from established clinical reality.

Platform and Structural Instability

The research also highlighted inconsistencies based on access methods. The same model (Perplexity-Sonar) yielded different results when accessed via API versus a web portal, with the latter showing significantly more inconsistency. Furthermore, structured JSON prompts forced models to be more conservative, increasing precision but further suppressing recall.

Conclusion and Future Directions

Current LLMs are not yet reliable for autonomous medical knowledge imputation. Their high success rate in finding single associations masks a fundamental inability to capture the breadth of clinical knowledge required for robust KGs.

Future work should focus on moving toward Retrieval-Augmented Generation (RAG) to ground outputs in clinical guidelines, and developing self-verification mechanisms where models check their work against medical ontologies before outputting relationships.

Summary updated

Abstract

Medical knowledge graphs (KGs) are essential for clinical decision support and biomedical research, yet they often exhibit incompleteness due to knowledge gaps and structural limitations in medical coding systems. This issue is particularly evident in treatment mapping, where coding systems such as ICD, Mondo, and ATC lack comprehensive coverage, resulting in missing or inconsistent associations between diseases and their potential treatments. To address this issue, we have explored the use of Large Language Models (LLMs) for imputing missing treatment relationships. Although LLMs offer promising capabilities in knowledge augmentation, their application in medical knowledge imputation presents significant risks, including factual inaccuracies, hallucinated associations, and instability between and within LLMs. In this study, we systematically evaluate LLM-driven treatment mapping, assessing its reliability through benchmark comparisons. Our findings highlight critical limitations, including inconsistencies with established clinical guidelines and potential risks to patient safety. This study serves as a cautionary guide for researchers and practitioners, underscoring the importance of critical evaluation and hybrid approaches when leveraging LLMs to enhance treatment mappings on medical knowledge graphs.
BibTeX
@misc{yao2025can,
  author = {Xinyu Yao and Aditya Sannabhadti and Holly M. Wiberg and Karmel S. Shehadeh and Rema Padman},
  title = {{Can LLMs Support Medical Knowledge Imputation? An Evaluation-Based Perspective}},
  year = {2025},
  eprint = {2503.22954},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2503.22954},
}
Figure