Abstract
The CARE-RAG (Clinical Assessment and Reasoning Evaluation for RAG) framework introduces a rigorous benchmark to evaluate whether Large Language Models (LLMs) truly reason with retrieved clinical data or merely rely on internal weights to produce plausible answers. Focusing on Written Exposure Therapy (WET) for PTSD, this study reveals a significant "reasoning gap": models often retrieve correct evidence but fail to adhere to strict clinical protocols. The findings highlight the necessity of evaluating logical entailment alongside accuracy in high-stakes medical applications.
Introduction
Retrieval-Augmented Generation (RAG) is frequently proposed as a solution to LLM hallucinations, under the assumption that access to "gold standard" clinical manuals ensures accurate output. However, in mental health, providing the right evidence is insufficient; models must interpret that evidence through structured protocols.
The authors identify a "reasoning gap" where a model might retrieve the correct passage but provide a clinically unsafe answer due to a failure in protocol logic. This is particularly critical in Written Exposure Therapy (WET), where therapeutic success depends on precise adherence to session steps. CARE-RAG asks: Do LLMs actually use retrieved information, or are they "lucky guessers" relying on pre-existing training data?
Methodology
The CARE-RAG framework evaluates models across three dimensions: context fidelity, reasoning complexity, and inference scoring.
Context Fidelity and Reasoning
Models were tested under three conditions:
- Right Context: Exact supporting evidence.
- Right Context + Noise: Evidence buried among adversarial distractors.
- Wrong Context: Misleading passages to test for over-reliance or hallucinations.
Tasks were stratified by reasoning demands (none, light, heavy). Beyond standard Accuracy, the researchers employed an LLM-as-judge to calculate a Fidelity Score, measuring the logical entailment between the model’s reasoning trace and the retrieved context.
Dataset
The study utilizes a newly curated dataset based on official WET manuals and PTSD clinical guidelines. It consists of 99 evaluation instances (Multiple Choice, Yes/No, and Open-Ended questions) vetted by a multi-panel team of psychologists. Unlike generic benchmarks, this dataset requires strict adherence to clinical logic.
Results and Key Insights
The evaluation of 20 state-of-the-art models, including reasoning-focused models like DeepSeek-R1 and Claude 3.5, revealed several critical trends.
The "Right Answer, Wrong Reason" Trap
A major finding was that models often achieve high accuracy on Multiple Choice Questions (MCQs) while maintaining low Inference Scores. This indicates that models frequently guess the correct answer based on internal weights without actually utilizing the provided clinical evidence. In the MCQ category, while several models reached 100% accuracy, their Inference Scores often hovered between 0.75 and 0.85, representing a 15-25% "reasoning deficit."
Clinical Nuance and Noise Sensitivity
Models frequently missed "gray area" nuances. For instance, many incorrectly suggested providing feedback on a patient's writing style during the first WET session, contradicting the protocol's focus on the trauma narrative. Furthermore, the introduction of "noisy" distractors significantly degraded performance, highlighting that "needle-in-a-haystack" retrieval remains a hurdle for clinical deployment.
Discussion and Future Work
While CARE-RAG provides a robust framework, it currently faces limitations such as potential LLM-as-judge bias and a sample size limited to WET protocols. Future work aims to expand the clinical scope and explore Agentic Context Engineering (ACE), where AI agents dynamically prioritize essential references from long-form documents.
Ultimately, the study underscores that until reasoning fidelity reaches near-perfection, LLMs in clinical settings require strict human guardrails and reasoning scaffolds to ensure patient safety.
Summary updated