Interpretation of 3D CNNs for Brain MRI Data Classification

Maxim Kan, Ruslan Aliev, Anna Rudenko, Nikita Drobyshev, Nikita Petrashen, Ekaterina Kondrateva, Maxim Sharaev, Alexander Bernstein, Evgeny Burnaev

Abstract

Deep learning, particularly 3D Convolutional Neural Networks (CNNs), offers significant potential for medical image analysis by processing raw data without extensive manual feature engineering. However, these models often operate as "black boxes," making interpretation difficult. This paper explores the classification of gender differences in brain MRI scans using a large-scale dataset of young healthy subjects. We compare three voxel-wise interpretation methods—Meaningful Perturbations, Grad-CAM, and Guided Backpropagation—to identify stable, neuroanatomically relevant patterns. Our results demonstrate that 3D CNNs achieve high classification accuracy and identify consistent structural differences that align with established neuroscience literature.

Introduction

While CNNs have shown success in identifying morphological differences in brain MRIs, most existing models lack granular, voxel-wise interpretation across the whole image. Understanding the specific features a model uses for classification is critical in clinical settings. This study addresses this gap by applying state-of-the-art interpretability tools to a 3D CNN trained on gender classification using the Human Connectome Project (HCP) dataset, which includes T1-weighted structural MRI scans from 1,113 healthy young adults.

Methodology

We implemented a lightweight 3D CNN architecture and compared its performance against traditional machine learning baselines, including Support Vector Machines (SVM), XGBoost, K-Nearest Neighbors (KNN), and Logistic Regression. The latter models were trained on "morphometry" features (e.g., cortical volume and surface area) extracted via FreeSurfer.

To interpret the CNN's decision-making process, we employed three distinct techniques:

  1. Meaningful Perturbations: Identifying critical regions by measuring the drop in model confidence when specific areas are blurred.
  2. Guided Backpropagation: Visualizing gradients to highlight voxels that positively influence classification.
  3. Grad-CAM: Using final-layer gradients to produce coarse localization maps of important regions.
Comparison of interpretation methods: (a) Grad-CAM, (b) Guided Backpropagation, and (c) Meaningful Perturbations.

Key Results

The 3D CNN achieved a mean classification accuracy of 0.92 ± 0.03, matching the performance of Logistic Regression trained on hand-crafted features and slightly outperforming the full-size image SVM (0.90). The baseline performance for other traditional models is summarized below:

Mean accuracy and standard deviation for baseline machine learning models (XGBoost, KNN, and Logistic Regression).

A critical finding was the high stability across interpretation methods. The DICE scores (measuring overlap) between the attention maps generated by the three techniques were above 0.93, indicating that the model identifies consistent biological signals regardless of the visualization method used.

Discussion and Neuroanatomical Validation

The regions identified as most discriminative by the CNN—including the Orbital Gyrus, Inferior Parietal Lobule, and Corticospinal tracts—align closely with existing literature on gender-related structural brain differences. Interestingly, the attention maps revealed patterns resembling the frontoparietal resting-state network, a hub for cognitive control. This suggests that structural variations in these functional hubs are primary drivers for the model's classification.

While the CNN performed well, the study noted that significant data downsampling (to $58 \times 70 \times 58$ tensors) was required to manage GPU memory. This compression likely limited the CNN's ability to outperform traditional models by losing fine-grained structural details.

Conclusion

This work demonstrates that 3D CNNs can reliably identify neuroanatomically meaningful patterns in brain MRI data. By providing a voxel-wise interpretation framework, we move closer to making deep learning models transparent and clinically useful. The authors have released the code as an open-source library to facilitate the application of these methods to other 3D medical imaging tasks, such as biomarker identification for Alzheimer’s disease or autism.

Summary updated

Abstract

Deep learning shows high potential for many medical image analysis tasks. Neural networks can work with full-size data without extensive preprocessing and feature generation and, thus, information loss. Recent work has shown that the morphological difference in specific brain regions can be found on MRI with the means of Convolution Neural Networks (CNN). However, interpretation of the existing models is based on a region of interest and can not be extended to voxel-wise image interpretation on a whole image. In the current work, we consider the classification task on a large-scale open-source dataset of young healthy subjects -- an exploration of brain differences between men and women. In this paper, we extend the previous findings in gender differences from diffusion-tensor imaging on T1 brain MRI scans. We provide the voxel-wise 3D CNN interpretation comparing the results of three interpretation methods: Meaningful Perturbations, Grad CAM and Guided Backpropagation, and contribute with the open-source library.
BibTeX
@inproceedings{kan2020interpretation,
  author = {Maxim Kan and Ruslan Aliev and Anna Rudenko and Nikita Drobyshev and Nikita Petrashen and Ekaterina Kondrateva and Maxim Sharaev and Alexander Bernstein and Evgeny Burnaev},
  title = {{Interpretation of 3D CNNs for Brain MRI Data Classification}},
  booktitle = {AIST (Supplement)},
  year = {2020},
  url = {https://openreview.net/forum?id=dUCkHgKcx6g},
}
Figure