Introduction
Large Language Models (LLMs) often struggle with clinical reasoning, frequently relying on surface-level pattern matching rather than formal medical logic. This failure mode, termed "schema collapse," is particularly problematic in clinical Natural Language Inference (NLI). In these tasks, models must determine if a clinical statement is entailed, contradicted, or neutral based on a provided premise. Standard prompting techniques like Chain-of-Thought (CoT) often fail to prevent "fluent but constraint-violating" reasoning because they lack the structural boundaries necessary for complex medical inference.
The CARENLI Framework
To address schema collapse, we introduce CARENLI (Compartmentalised Agentic Reasoning for Clinical NLI), a modular agentic framework that enforces domain-specific reasoning schemas. The system routes clinical queries into specialized "compartments" designed for specific reasoning families: Causal, Compositional, Epistemic, or Risk.
The framework employs a four-stage pipeline:
- Router: Identifies the dominant reasoning signature and assigns the task to the appropriate compartment.
- Solver: Executes reasoning rules specific to the assigned family (e.g., verifying control groups for causal claims).
- Verifier: Audits the solver's output for factual grounding and compliance with expected reasoning schemas.
- Refiner: Corrects errors identified by the verifier to produce the final label.
Experimental Evaluation
The framework was evaluated using an extended version of the CTNLI (Clinical Trial NLI) benchmark, consisting of 200 expert-validated instances across four reasoning types. Testing involved four backbone models: GPT-5.1, GPT-4.1, GPT-4o-mini, and DeepSeek R1.
Results and Discussion
CARENLI significantly outperformed standard prompting methods, increasing mean macro accuracy from 23.1% (direct baseline) to 56.8%, a gain of approximately 34 percentage points.
Key findings include:
- Reasoning Variability: Performance varied significantly by reasoning type. Risk State Abstraction was the most successful (92.2% accuracy), while Compositional Grounding remained the most challenging (~25.6%).
- The Routing Bottleneck: System success is highly sensitive to the initial routing; misclassification leads to a sharp drop in accuracy (from ~71% to ~21%).
- Latent Competence: The results suggest that LLMs possess latent medical knowledge but require "procedural scaffolding" to deploy it correctly.
- Scale Limitations: Even frontier models like GPT-5.1 and DeepSeek R1 exhibit systematic failures under standard prompting, indicating that model scaling alone does not resolve schema collapse.
Conclusion
CARENLI demonstrates that compartmentalizing reasoning into specialized agentic roles can recover clinical accuracy that is otherwise lost to schema collapse. While compositional reasoning remains a hurdle, the framework provides auditable reasoning traces that are superior to free-form CoT for clinical applications. Future work will focus on expanding reasoning families and testing on real-world electronic health record (EHR) data.
Summary updated