Spatial Sequence Attention Network for Schizophrenia Classification from Structural Brain MR Images

Nagur Shareef Shaik, Teja Krishna Cherukuri, Vince D. Calhoun, Dong Hye Ye

Abstract

Schizophrenia (SZ) is a chronic mental disorder associated with subtle morphological changes in brain structure, particularly in gray matter density. Automated classification using deep learning faces challenges due to limited clinical data and the difficulty of capturing localized spatial interactions. This paper presents SSANet, a novel architecture that combines transfer learning with a Spatial Sequence Attention (SSA) mechanism. By utilizing Convolutional LSTMs to process 3D brain volumes as sequences, the model effectively captures intricate spatial relationships. Experimental results on the FBIRN dataset demonstrate that SSANet outperforms standard benchmarks, achieving an accuracy of 75.12% while providing biological validation through attention heatmaps.

Introduction

Schizophrenia is characterized by complex neuroanatomical variations that are often too minute for manual detection by radiologists. While deep learning, particularly Convolutional Neural Networks (CNNs), has revolutionized medical imaging, its application to psychiatric diagnosis is hindered by two primary factors: data scarcity and feature localization. Clinical datasets for Schizophrenia are typically small, leading to overfitting in complex models. Furthermore, standard CNNs often prioritize holistic features, potentially overlooking the localized spatial interactions across different brain volumes that are critical for accurate diagnosis.

To address these hurdles, we introduce the Spatial Sequence Attention Network (SSANet). This model leverages transfer learning to mitigate data scarcity and introduces a specialized attention mechanism designed to interpret 3D structural MRI (sMRI) data as a spatial sequence.

Methodology

The SSANet architecture is composed of three primary phases: feature extraction, spatial attention, and classification.

1. Transfer Learning for Feature Extraction

To overcome the limitations of small clinical datasets, the model employs a 3D DenseNet121 pre-trained on large-scale medical imaging data. Rather than utilizing the final classification layer, feature maps are extracted from the final convolutional block. This approach allows the model to leverage sophisticated pre-existing knowledge of brain morphology, providing a robust foundation for downstream tasks.

2. Spatial Sequence Attention (SSA) Module

The core innovation of this work is the SSA module, which treats 3D feature maps as a sequence of spatial slices. It utilizes a Convolutional Long Short-Term Memory (ConvLSTM) layer. While LSTMs are traditionally applied to temporal data, here they are repurposed to capture dependencies across spatial volumes. This enables the model to "attend" to relevant regions—such as the frontal and temporal lobes—and understand their inter-relationships within the 3D coordinate system.

3. Classification

The attended features are processed through a regularized Feed Forward Neural Network (FFNN). This network consists of two dense layers (512 and 256 neurons) utilizing the Gaussian Error Linear Unit (GELU) activation function to produce the final diagnostic classification.

Experimental Setup

Dataset

The study utilized structural MRI (sMRI) data from the Function Biomedical Informatics Research Network (FBIRN). The dataset included 186 participants:

  • 82 individuals diagnosed with Schizophrenia.
  • 104 healthy controls.

All images were downsampled to $121 \times 145 \times 121$ voxels to ensure computational efficiency without significant loss of morphological detail.

Results and Discussion

SSANet was evaluated using 5-fold cross-validation and compared against several baseline models, including standard DenseNet, XGBoost, and a Squeeze-and-Excitation (SENet) attention baseline.

Method Accuracy (Acc) Precision (Prec) Recall $F_1$ Score
DenseNet 70.00% - - -
XGBoost 71.51% - - -
SENet (Baseline Attention) 72.80% 77.84% 68.06% 66.79%
SSANet (Ours) 75.12% 77.22% 72.76% 71.91%

Performance Analysis

SSANet achieved the highest accuracy (75.12%) and demonstrated superior balance between Recall and $F_1$ score compared to the SENet approach. This suggests that the spatial sequence approach is more reliable at identifying true positive cases of Schizophrenia.

Biological Validation

A qualitative evaluation using attention heatmaps revealed that the model naturally focused on the frontal and temporal lobes. This finding is significant as it aligns with established clinical literature regarding the brain regions most impacted by Schizophrenia. This provides a layer of interpretability, suggesting the model's decisions are based on biologically relevant features rather than noise.

Conclusion and Future Work

SSANet demonstrates that combining transfer learning with spatial sequence attention can significantly improve the classification of Schizophrenia from sMRI scans, even with limited data. However, several avenues for future research remain:

  1. Generalization: Testing the model on larger, more diverse datasets to ensure robustness across different populations and scanner types.
  2. Multi-modal Integration: Incorporating functional MRI (fMRI) and genetic data (such as SNPs) to capture the multi-faceted nature of the disorder.
  3. Interpretability: Enhancing the transparency of the model's reasoning to facilitate its adoption in clinical decision-support systems.

Summary updated

BibTeX
@inproceedings{shaik2024spatial,
  author = {Nagur Shareef Shaik and Teja Krishna Cherukuri and Vince D. Calhoun and Dong Hye Ye},
  title = {{Spatial Sequence Attention Network for Schizophrenia Classification from Structural Brain MR Images}},
  booktitle = {ISBI},
  year = {2024},
  doi = {10.1109/isbi56570.2024.10635528},
}
Figure