An explainable three dimensional framework to uncover learning patterns: A unified look in variable sulci recognition

Michail Mamalakis, Héloïse de Vareilles, Atheer Al-Manea, Samantha C. Mitchell, Ingrid Agartz, Lynn Egeland Mørch-Johnsen, Jane R. Garrison, Jon S. Simons, Pietro Lio, John Suckling, Graham K. Murray

TL;DR

This paper introduces a novel 3D explainable AI (XAI) framework to understand how deep learning models identify subtle anatomical variations in the brain, specifically the paracingulate sulcus (PCS). By combining statistical shape analysis with advanced XAI techniques (GradCam and SHAP) in a 3D space, the framework provides more accurate and interpretable insights into brain development and its relation to conditions like psychosis.

What Problem Does This Paper Address?

Neuroimaging, particularly the analysis of brain structures like sulci, presents a significant challenge for AI due to the high dimensionality and complexity of the data. Existing XAI methods, often designed for 2D data, struggle to capture the nuances of 3D brain structures, leading to potentially inaccurate or misleading explanations. This lack of robust 3D explainability hinders our ability to trust AI models and uncover novel patterns in brain development and its link to neurological and psychiatric disorders. Specifically, the paracingulate sulcus (PCS) is a highly variable structure whose presence is associated with conditions like psychosis, making its accurate identification and understanding crucial.

Methods

The core of this work is the development of an XAI 3D-Framework that integrates three key components:

  • Statistical Features (Shape): Principal Component Analysis (PCA) is used to reduce the dimensionality of 3D brain inputs (sulcal skeleton and grey-white surface) and capture global feature importance patterns. This provides a cohort-specific understanding of variability.
  • Local XAI Methods (GradCam and SHAP): These established XAI techniques are extended into 3D to provide local explanations for the AI's predictions. GradCam highlights important regions based on feature activations, while SHAP quantifies the contribution of each feature to the prediction.
  • Unified Global Explanation: The framework combines the insights from statistical features and local XAI methods through a weighted averaging approach. This fusion aims to reduce inter-method variability and enhance the faithfulness and reliability of the global explanations.

The study also employed two deep learning architectures for the classification task: a simple 3D Convolutional Neural Network (simple-3D-CNN) and a two-headed attention layer network (simple-3D-MHL).

Datasets

The study utilized the TOP-OSLO cohort, comprising 596 structural MRI scans. This cohort includes individuals with schizophrenia spectrum disorders, bipolar disorder, and unaffected controls. The data was pre-processed to extract grey-white surfaces and sulcal skeletons.

Key Insights and Ideas

  • 3D Explainability is Crucial for Neuroimaging: The paper strongly advocates for and demonstrates the necessity of 3D-specific XAI methods to accurately interpret complex brain structures.
  • Fusion of Statistical and Model-Driven Explanations: Combining cohort-level statistical shape information with AI model-derived explanations (GradCam, SHAP) leads to more robust and trustworthy insights.
  • Reduced Inter-Method Variability: By integrating multiple XAI techniques, the framework mitigates the issue of different methods highlighting different features, leading to more consistent and reliable explanations.
  • Uncovering Developmental Insights: The framework's ability to identify key brain regions associated with PCS presence/absence suggests potential co-variations and shared genetic or developmental mechanisms influencing brain development and its link to psychiatric conditions.
  • PCS Variability and Psychosis Link: The study reinforces the importance of the PCS's variability and its association with conditions like psychosis, providing a new tool to investigate these links.

Key Results

  • The proposed XAI 3D-Framework demonstrated superior faithfulness scores compared to standalone GradCam and SHAP methods in both left and right hemispheres, exceeding 0.21 in the left hemisphere.
  • The framework achieved competitive complexity scores, indicating a balance between interpretability and comprehensiveness.
  • The models achieved classification accuracies of around 73% in the left hemisphere and 58-63% in the right hemisphere for PCS detection.
  • Key brain regions consistently highlighted by the framework for PCS classification included the cingulate gyrus, posterior temporal region, and medial/inferior frontal cortices, suggesting potential neurodevelopmental links.

Limitations and Future Work

  • Data from a Single Cohort: The study acknowledges the importance of incorporating data from multiple cohorts to generalize findings and further validate the framework.
  • Extension to Other Conditions: The authors plan to extend the framework's application to other neurological conditions like schizophrenia and bipolar disorder, leveraging external datasets.
  • Advanced Dimensionality Reduction: Future work may explore advanced techniques like t-SNE for non-linear dimensionality reduction to gain deeper insights into relationships.
  • Broader Application: The framework is positioned as a foundational tool for broader applications in medical imaging and other complex computer vision tasks, aiming to advance neuroanatomy and developmental mechanism research.

Summary updated

Abstract

The significant features identified in a representative subset of the dataset during the learning process of an artificial intelligence model are referred to as a 'global' explanation. 3D global explanations are crucial in neuroimaging, where a complex representational space demands more than basic 2D interpretations. However, current studies in the literature often lack the accuracy, comprehensibility, and 3D global explanations needed in neuroimaging and beyond. To address this gap, we developed an explainable artificial intelligence (XAI) 3D-Framework capable of providing accurate, low-complexity global explanations. We evaluated the framework using various 3D deep learning models trained on a well-annotated cohort of 596 structural MRIs. The binary classification task focused on detecting the presence or absence of the paracingulate sulcus, a highly variable brain structure associated with psychosis. Our framework integrates statistical features (Shape) and XAI methods (GradCam and SHAP) with dimensionality reduction, ensuring that explanations reflect both model learning and cohort-specific variability. By combining Shape, GradCam, and SHAP, our framework reduces inter-method variability, enhancing the faithfulness and reliability of global explanations. These robust explanations facilitated the identification of critical sub-regions, including the posterior temporal and internal parietal regions, as well as the cingulate region and thalamus, suggesting potential genetic or developmental influences. Our XAI 3D-Framework leverages global explanations to uncover the broader developmental context of specific cortical features. This approach advances the fields of deep learning and neuroscience by offering insights into normative brain development and atypical trajectories linked to mental illness, paving the way for more reliable and interpretable AI applications in neuroimaging.
BibTeX
@article{mamalakis2026explainable,
  author = {Michail Mamalakis and Héloïse de Vareilles and Atheer Al-Manea and Samantha C. Mitchell and Ingrid Agartz and Lynn Egeland Mørch-Johnsen and Jane R. Garrison and Jon S. Simons and Pietro Lio and John Suckling and Graham K. Murray},
  title = {{An explainable three dimensional framework to uncover learning patterns: A unified look in variable sulci recognition}},
  journal = {Artif. Intell. Medicine},
  year = {2026},
  doi = {10.1016/j.artmed.2025.103286},
}
Figure