Rethinking Explainable Disease Prediction: Synergizing Accuracy and Reliability via Reflective Cognitive Architecture

Zhijing Shao, Haiyang Shen, Mugeng Liu, Gecheng Fu, Yaoqi Guo, Yanfeng Wang, Yun Ma

Abstract

In clinical decision-making, predictive models face a persistent trade-off: accurate models are often opaque "black boxes," while interpretable methods frequently lack predictive precision or statistical grounding. In this paper, we challenge this dichotomy, positing that high predictive accuracy and high-quality descriptive explanations are not competing goals but synergistic outcomes of a deep, first-hand understanding of data. We propose the Reflective Cognitive Architecture (RCA), a novel framework designed to enable Large Language Models (LLMs) to learn directly from tabular data through experience and reflection. RCA integrates two core mechanisms: an iterative rules optimization process that refines logical argumentation by learning from prediction errors, and a distribution-aware rules check that grounds this logic in global statistical evidence to ensure robustness. We evaluated RCA against over 20 baselines across diverse medical datasets, including a proprietary real-world Catheter-Related Thrombosis (CRT) cohort and two large-scale datasets. The results confirm that RCA achieves state-of-the-art predictive performance and superior robustness to data noise while simultaneously generating clear, logical, and evidence-based explanatory statements.

Introduction

In clinical settings, AI faces a difficult choice. Traditional machine learning models (e.g., linear regression, decision trees) are interpretable but often require experts to manually parse statistical outputs, which increases cognitive load. Conversely, deep learning models offer high accuracy but function as opaque "black boxes."

Large Language Models (LLMs) offer a potential middle ground by providing natural language explanations. However, standard LLMs often lack a granular understanding of specific tabular datasets, leading to "statistically ungrounded" explanations that sound plausible but lack empirical support. Furthermore, LLM agents that rely on external tools (like Python interpreters) often abstract the data away, preventing the model from developing the instance-level intuition necessary for high-quality clinical reasoning. We argue that accuracy and explainability should be synergistic: a model that truly understands the data distribution will naturally perform better and explain its reasoning more clearly.

The Reflective Cognitive Architecture (RCA)

The Reflective Cognitive Architecture (RCA) is a framework that enables LLMs to learn from tabular data through a cycle of experience and reflection, mimicking the way a human analyst develops clinical intuition.

The Reflective Cognitive Architecture (RCA) workflow, illustrating the iterative self-reflection process during training and the rule-based inference phase.

As shown in Figure 1, the RCA pipeline consists of three core stages:

  1. Data Narrative & Distribution Extraction: Tabular patient data is converted into unstructured text. Simultaneously, the system extracts the global data distribution (means, percentiles, frequencies) from the training set to serve as a statistical ground truth.
  2. Iterative Rules Optimization (Bottom-Up Learning): A prediction LLM makes diagnoses based on a dynamic rule base. When an error occurs, a "Reflection LLM" analyzes the misclassified samples and proposes new logical rules to correct the mistake.
  3. Distribution-aware Rules Check (Top-Down Safeguard): To prevent overfitting to outliers, a "Checking LLM" audits proposed rules against the global data distribution. If a rule contradicts the broader statistical reality of the dataset, it is generalized or discarded.

Experimental Results

We evaluated RCA on five datasets: a proprietary Catheter-Related Thrombosis (CRT) dataset, public Diabetes and Heart Disease benchmarks, and two large-scale datasets (CRT_ex and a 70,000-patient Cardiovascular Disease set) to test scalability.

Predictive Performance and Robustness

RCA (using GPT-4.1 or Qwen2.5) outperformed 25 baselines, including CatBoost and advanced reasoning models like DeepSeek-R1. On the Heart Disease dataset, RCA+GPT-4.1 achieved 74.6% accuracy. Crucially, RCA demonstrated exceptional robustness; when subjected to data stress—such as removing features or injecting 10% extreme outliers—RCA maintained stable performance while traditional ML models saw significant degradation.

Human Evaluation and Cognitive Load

In blind tests, three clinical experts preferred RCA's explanations over the strongest baselines 72% to 89% of the time. The framework was specifically praised for its logical argumentation and adherence to evidence-based medicine.

Comparison of Cognitive Load vs. Accuracy and MCC across various CRT dataset conditions, highlighting RCA's superior performance and reliability.

As illustrated in Figure 6, RCA consistently achieves higher accuracy and Matthews Correlation Coefficient (MCC) compared to baselines across various data conditions (missing values, no GLR), while maintaining high scores in human-centric metrics like cognitive load reduction.

Key Insights

  • Synergy of Accuracy and Explainability: By forcing the model to build a statistically grounded rule base, RCA naturally produces both higher accuracy and more reliable explanations.
  • Global Context Prevents Hallucination: Providing the LLM with global statistical distributions acts as a vital "sanity check," preventing the model from over-interpreting outliers.
  • Implicit Sub-population Discovery: RCA organically learned to stratify patients, developing distinct diagnostic rules for different clinical profiles (e.g., high vs. low baseline D-dimer levels).

Conclusion

RCA demonstrates that LLMs can overcome the accuracy-interpretability trade-off in healthcare by engaging in iterative reflection grounded in global data statistics. While the architecture requires more computational overhead than simple tabular models, its ability to provide clinician-approved, evidence-based reasoning makes it a significant step toward reliable AI in medicine.

Summary updated

Abstract

In clinical decision-making, predictive models face a persistent trade-off: accurate models are often opaque "black boxes," while interpretable methods frequently lack predictive precision or statistical grounding. In this paper, we challenge this dichotomy, positing that high predictive accuracy and high-quality descriptive explanations are not competing goals but synergistic outcomes of a deep, first-hand understanding of data. We propose the Reflective Cognitive Architecture (RCA), a novel framework designed to enable Large Language Models (LLMs) to learn directly from tabular data through experience and reflection. RCA integrates two core mechanisms: an iterative rules optimization process that refines logical argumentation by learning from prediction errors, and a distribution-aware rules check that grounds this logic in global statistical evidence to ensure robustness. We evaluated RCA against over 20 baselines - ranging from traditional machine learning to advanced reasoning LLMs and agents - across diverse medical datasets, including a proprietary real-world Catheter-Related Thrombosis (CRT) cohort. Crucially, to demonstrate real-world scalability, we extended our evaluation to two large-scale datasets. The results confirm that RCA achieves state-of-the-art predictive performance and superior robustness to data noise while simultaneously generating clear, logical, and evidence-based explanatory statements, maintaining its efficacy even at scale. The code is available at https://github.com/ssssszj/RCA.
BibTeX
@misc{shao2025rethinking,
  author = {Zhijing Shao and Haiyang Shen and Mugeng Liu and Gecheng Fu and Yaoqi Guo and Yanfeng Wang and Yun Ma},
  title = {{Rethinking Explainable Disease Prediction: Synergizing Accuracy and Reliability via Reflective Cognitive Architecture}},
  year = {2025},
  eprint = {2509.21266},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2509.21266},
}
Figure