XAI-MeD: Explainable Knowledge Guided Neuro-Symbolic Framework for Domain Generalization and Rare Class Detection in Medical Imaging

Midhat Urooj, Ayan Banerjee, Sandeep GuptaarXiv 2026 · DOI · arXiv

TL;DR

XAI-MeD is a neuro-symbolic framework that bridges the gap between "black-box" deep learning and structured medical expertise. By integrating clinical rules into the model's decision-making process, it achieves a 10% boost in detecting rare diseases and significantly improves performance when moving between different hospitals (domain generalization), all while providing human-readable explanations.

What Problem Does This Paper Address?

Deep learning (DL) has revolutionized medical imaging, but it faces a "trust gap" in clinical settings. Three major hurdles persist:

  1. Interpretability: Post-hoc methods like heatmaps (Grad-CAM) often show where a model looks but not why it made a specific clinical judgment.
  2. Rare-Class Learning: Because life-threatening pathologies are often infrequent in datasets, standard models tend to ignore them in favor of more common "normal" cases.
  3. Domain Generalization: A model trained on data from Hospital A often fails at Hospital B due to different scanner brands, imaging protocols, or patient demographics.

The authors argue that these failures happen because DL models rely purely on statistical correlations in pixels rather than the underlying "logic" of medicine.

Methods

The core of XAI-MeD is its neuro-symbolic architecture, which runs two branches in parallel:

  • The Neural Branch: Standard deep learning (CNNs or Vision Transformers) to extract high-dimensional features from raw images.
  • The Symbolic Branch: An "Expert Knowledge Processor" that encodes clinical expertise as logical propositions (e.g., "If there is a large cluster of gray matter activation AND high-frequency power, THEN it is a Seizure Onset Zone").

To fuse these branches, the authors introduced the EKSAII (Expert Knowledge and Supervised AI Integration) algorithm. This is an adaptive routing mechanism inspired by Hunt’s algorithm. It uses two key metrics to decide which "expert" (neural or symbolic) should handle a specific data point:

  • Entropy Imbalance Gain (EIG): Quantifies how well a classifier handles rare classes.
  • Rare-Class Gini Index: Measures the "purity" of a prediction to manage high variability within a single disease grade.

Finally, the framework feeds the symbolic features and the final diagnosis into a Large Language Model (GPT-4) to generate a natural language explanation that aligns with how a doctor actually speaks.

Datasets

The framework was validated across ten multicenter datasets covering two distinct clinical tasks:

  1. Seizure Onset Zone (SOZ) Localization: Using resting-state fMRI (rs-fMRI) data from the Phoenix Child Health Center (PCH) and the University of North Carolina (UNC).
  2. Diabetic Retinopathy (DR) Grading: Using retinal fundus images from four major public datasets: APTOS, EyePACS, Messidor-1, and Messidor-2.

Key Insights and Ideas

  • Symbolic Logic as a Regularizer: One of the most interesting findings is that the symbolic branch acts as a "guardrail." When the neural branch encounters a "domain shift" (e.g., a new type of camera), the symbolic rules—which are based on universal medical truths—keep the model's performance stable.
  • Beyond Heatmaps: Unlike traditional XAI that just highlights pixels, XAI-MeD provides "machine-checkable" rules. This means a clinician can see exactly which medical criteria (like "vessel dilation" or "exudate density") were met.
  • The "Expert Selection" Paradigm: Instead of just averaging the neural and symbolic outputs, the EKSAII algorithm treats them as a competitive pool of experts, dynamically routing the hardest cases (rare classes) to the most "competent" branch.

Key Results

  • Rare Class Detection: XAI-MeD achieved a 10% improvement in F1-score for rare disease classes compared to state-of-the-art DL baselines.
  • Generalization: In cross-domain tests (training on one hospital, testing on another), the framework showed a 6% gain in accuracy, maintaining robustness where pure DL models crashed.
  • Clinical Efficiency: In the seizure localization task, the framework reduced the manual expert review effort by 84.2% (from 110 components down to just 18).
  • Top-Tier Performance: For Diabetic Retinopathy grading, the system reached a peak accuracy of 84%, outperforming complex domain generalization methods like Fishr and SPSD-ViT.

Limitations and Future Work

  • Dependence on Auxiliary Models: To extract symbolic features (like counting hemorrhages), the framework currently relies on "pre-processor" models like YOLO or U-Net. If these auxiliary models fail, the symbolic branch receives "garbage in."
  • Finite Knowledge Base: The symbolic module relies on a fixed set of expert-defined rules. It cannot yet "discover" new clinical markers on its own.
  • Scaling Knowledge Engineering: Manually encoding rules for every possible disease is labor-intensive. The authors suggest that future work could involve using LLMs to automatically induce these rules from massive medical corpora.

Summary updated

Abstract

Explainability domain generalization and rare class reliability are critical challenges in medical AI where deep models often fail under real world distribution shifts and exhibit bias against infrequent clinical conditions This paper introduces XAIMeD an explainable medical AI framework that integrates clinically accurate expert knowledge into deep learning through a unified neuro symbolic architecture XAIMeD is designed to improve robustness under distribution shift enhance rare class sensitivity and deliver transparent clinically aligned interpretations The framework encodes clinical expertise as logical connectives over atomic medical propositions transforming them into machine checkable class specific rules Their diagnostic utility is quantified through weighted feature satisfaction scores enabling a symbolic reasoning branch that complements neural predictions A confidence weighted fusion integrates symbolic and deep outputs while a Hunt inspired adaptive routing mechanism guided by Entropy Imbalance Gain EIG and Rare Class Gini mitigates class imbalance high intra class variability and uncertainty We evaluate XAIMeD across diverse modalities on four challenging tasks i Seizure Onset Zone SOZ localization from rs fMRI ii Diabetic Retinopathy grading across 6 multicenter datasets demonstrate substantial performance improvements including 6 percent gains in cross domain generalization and a 10 percent improved rare class F1 score far outperforming state of the art deep learning baselines Ablation studies confirm that the clinically grounded symbolic components act as effective regularizers ensuring robustness to distribution shifts XAIMeD thus provides a principled clinically faithful and interpretable approach to multimodal medical AI.
BibTeX
@misc{urooj2026xai,
  author = {Midhat Urooj and Ayan Banerjee and Sandeep Gupta},
  title = {{XAI-MeD: Explainable Knowledge Guided Neuro-Symbolic Framework for Domain Generalization and Rare Class Detection in Medical Imaging}},
  year = {2026},
  eprint = {2601.02008},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2601.02008},
}
Figure