Abstract
As medical Large Language Models (LLMs) proliferate, ensuring their reliability for patient self-diagnosis is critical. This paper introduces the Self-Diagnostic Atomic Knowledge (SDAK) benchmark, a novel framework designed to evaluate the fundamental medical knowledge of Chinese LLMs. By decomposing medical information into "atomic knowledge items" and employing a contrastive factual/counterfactual evaluation, the study reveals significant "sycophancy" in current models—where models prioritize user agreement over factual accuracy. Our analysis of various Chinese medical LLMs shows that while scaling improves performance, a substantial gap remains compared to state-of-the-art general models like GPT-4.
Introduction
The rise of medical LLMs offers promising avenues for preliminary health consultations. However, existing evaluation paradigms—standardized exams, NLP tasks, or subjective GPT-4 judging—often fail to capture whether a model truly "knows" a fact or is merely generating plausible-sounding text. Standardized exams reflect professional expertise rather than patient-centric queries, while GPT-4 evaluations can be biased. Crucially, these methods do not isolate atomic knowledge (the smallest units of medical fact) from reasoning capabilities.
The SDAK Benchmark Methodology
To address these gaps, we developed the SDAK benchmark, focusing on the indivisible units of information required for accurate self-diagnosis. The construction process involves four key stages:
- Thematic Analysis: Identifying 17 "atomic types" (e.g., Disease-Symptom, Medicine-Effect) based on real-world user search intents from the KUAKE-QIC dataset.
- Knowledge Extraction: Sourcing high-quality facts from professional Chinese medical platforms like Xiaohe and 120ask.
- Item Construction: Generating structured medical content and pairing factual claims with counterfactual counterparts.
- Manual Verification: Ensuring the accuracy and objectivity of the generated items through expert review.
Evaluation Framework
A core innovation of SDAK is the Contrastive Evaluation method. To mitigate "sycophancy"—the tendency of models to agree with user prompts regardless of truth—models are presented with both factual and counterfactual claims. This forces the model to demonstrate consistent knowledge rather than simple agreement.
We employ three primary metrics to assess model performance:
- Instruction Following Rate (IFR): Measures if the model adheres to the requested "correct/incorrect" response format.
- Factual Accuracy (FactAcc): The model must correctly identify the factual claim as "correct" AND the counterfactual one as "incorrect."
- Accuracy Reliability (AccR): A manual audit to verify if the model's provided reasoning is logically sound and supports its answer.
Key Findings and Results
Our computational analysis of several Chinese medical LLMs yielded several critical insights:
The Sycophancy Problem
The most significant challenge identified is sycophancy. Many models indiscriminately support user-suggested claims. If a prompt suggests a false medical fact, the model often confirms it to be "helpful," highlighting a dangerous lack of grounded truthfulness in current medical-specific tuning.
The Domain Adaptation Tax
Fine-tuning general LLMs on medical data often results in a "tax" on general capabilities. While general-purpose models like Qwen or Baichuan maintained near 100% IFR, specialized medical models saw declines to as low as 76%, suggesting that domain-specific alignment can degrade basic instruction-following discipline.
Performance Benchmarks
- GPT-4 Superiority: GPT-4 remains the gold standard, achieving a FactAcc of 65.42%.
- Medical LLM Lag: The highest-performing open-source Chinese medical model, Zhongjing, reached only 24.78% FactAcc in zero-shot settings.
- Data Quality: Training on data "distilled" from advanced models (like GPT-4) proved more effective for knowledge retention than using messy, real-world doctor-patient transcripts.
Conclusion
The SDAK benchmark provides a rigorous tool for quantifying the factual foundations of medical LLMs. Our results underscore the need for training strategies that prioritize truthfulness over agreeableness, such as specialized Reinforcement Learning from Human Feedback (RLHF). Future work will focus on expanding the knowledge base and exploring techniques to preserve instruction-following capabilities during medical domain adaptation.
Summary updated