Abstract
Large Language Models (LLMs) demonstrate impressive recall of clinical facts but often fail to apply this knowledge in complex medical reasoning tasks. This study identifies a "Knowledge-Reasoning Dissociation," where models achieve high accuracy in factual verification (92%) yet perform poorly in clinical logic puzzles (25%). We introduce the Clinical Trial Natural Language Inference (CTNLI) benchmark and Ground Knowledge and Meta-Level Reasoning Verification (GKMRV) probes to decouple factual knowledge from logical inference. Our findings suggest that current scaling paradigms improve "encyclopedic" recall without necessarily developing the structured internal representations required for safe clinical decision-making.
Introduction
A persistent optimism in AI research suggests that increasing model scale and data volume will eventually yield human-like reasoning. In high-stakes domains like medicine, it is often assumed that a model's ability to recall a medical fact implies an ability to utilize it. However, LLMs may rely on surface-level pattern completion rather than genuine understanding. For instance, a model might recommend a treatment based on its resemblance to training data rather than an understanding of causal mechanics, such as the necessity of a control group. This paper investigates whether LLM failures in clinical settings stem from "ignorance" (lack of knowledge) or "illogic" (failure to reason).
Methodology
To investigate the gap between knowledge and reasoning, we developed a framework consisting of two primary components: the CTNLI benchmark and GKMRV probes.
The CTNLI Benchmark
The Clinical Trial Natural Language Inference (CTNLI) benchmark categorizes clinical reasoning into four "families":
- Causal Attribution: Distinguishing correlation from causation.
- Compositional Grounding: Integrating multiple constraints (e.g., patient age, dosage, and renal function).
- Epistemic Verification: Evaluating claims based on evidence rather than authority.
- Risk State Abstraction: Weighing the severity and probability of clinical outcomes.
GKMRV Probes
The Ground Knowledge and Meta-Level Reasoning Verification (GKMRV) tool serves as a diagnostic probe. For each reasoning task, a paired True/False question is presented to the model regarding the underlying rule or fact required for the main task. This allows for a direct comparison between what the model "knows" and how it "reasons."
Data Generation
Unlike standard datasets prone to training data leakage, CTNLI uses formalized, parameterized templates to generate controlled examples. This ensures models are tested on their ability to process specific logical structures rather than memorized clinical trial results.
Results and Key Insights
Testing across six top-tier models (including GPT-4o, OpenAI o3, and DeepSeek R1) revealed several critical findings:
- The Dissociation Phenomenon: Models frequently passed factual checks (GKMRV) while failing the associated reasoning tasks. Mean accuracy for knowledge was 0.918, whereas reasoning accuracy plummeted to 0.25.
- Systematic Heuristics: Models often employ flawed shortcuts. In causal tasks, they frequently equate numerical effect size with proof of efficacy, ignoring the absence of control groups.
- Compositional Collapse: Models struggled significantly with "Compositional Grounding," averaging only 4% accuracy. While they could identify individual factors (e.g., a drug's safety profile or a patient's condition), they failed to reason about their interaction.
- Consistency in Error: Models exhibited high internal consistency (87%), meaning they are often confidently and repeatedly wrong—a trait particularly dangerous in clinical environments.
- Limited Impact of CoT: Chain-of-Thought (CoT) prompting yielded only a marginal improvement (+0.03), suggesting that verbalizing steps does not compensate for a fundamental lack of internal logical structure.
Conclusion and Future Directions
The observed dissociation suggests that current LLM training focuses on factual density at the expense of logical robustness. Future research should move beyond simple data scaling toward:
- Neuro-symbolic Integration: Combining LLM flexibility with formal symbolic logic engines.
- Representation Disentanglement: Developing training methods that separate factual recall from inferential machinery.
- Counterfactual Modeling: Building models capable of simulating "what if" scenarios, which are essential for accurate medical risk assessment.
By addressing these structural limitations, we can move closer to AI systems that are not just knowledgeable, but truly capable of safe and reliable clinical reasoning.
Summary updated