Residual and plain convolutional neural networks for 3D brain MRI classification

Sergey Korolev, Amir Safiullin, Mikhail Belyaev, Yulia Dodonova

Introduction

The diagnosis of Alzheimer’s Disease (AD) through neuroimaging has traditionally relied on complex, multi-stage "handcrafted" pipelines. These standard approaches typically involve labor-intensive preprocessing steps such as skull stripping, tissue segmentation (e.g., separating gray matter from white matter), and manual feature extraction before data can be processed by a classifier like a Support Vector Machine (SVM). Such pipelines require significant domain expertise and are prone to human error or bias.

This study explores the viability of using 3D Convolutional Neural Networks (CNNs) to classify AD from structural brain MRI scans with minimal manual intervention. By leveraging end-to-end deep learning, the researchers aim to bypass traditional feature engineering, allowing the models to learn relevant diagnostic features directly from 3D spatial data. Despite the challenges of small neuroimaging datasets, this research demonstrates that modern architectures can achieve competitive results while discovering clinically relevant biomarkers autonomously.

Methods

The researchers evaluated two distinct 3D CNN architectures to determine the impact of network depth and connectivity on classification performance:

  1. VoxCNN: A "plain" feedforward network inspired by the VGG architecture. It consists of four volumetric convolutional blocks designed to extract hierarchical features, followed by fully connected layers for classification. To mitigate overfitting—a common issue with small medical datasets—the model incorporates Batch Normalization and Dropout ($p=0.7$).
  2. ResNet: Based on the "VoxResNet" architecture, this model is significantly deeper, comprising 21 layers. It utilizes residual (skip) connections and identity mappings to facilitate gradient flow during training. This architecture is specifically designed to handle the vanishing gradient problem often encountered in deep networks trained on limited data.
Architectures of VoxCNN (plain) and ResNet (residual) used for 3D MRI classification.

A critical technical innovation in this study is the implementation of balanced batch sampling. Given the small sample size, random training batches often suffer from class imbalance, which can destabilize the learning process. The authors modified the iteration logic to ensure each batch contains a representative mixture of classes, providing a more stable gradient signal.

Datasets

The study utilized structural MRI scans from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) database. The dataset comprised 231 images categorized into four classes: Alzheimer’s Disease (AD), Late Mild Cognitive Impairment (LMCI), Early Mild Cognitive Impairment (EMCI), and Normal Control (NC).

The input data consisted of 3D tensors with dimensions of $110 \times 110 \times 110$. While the images underwent standard ADNI preprocessing—including spatial normalization and masking—the authors intentionally avoided the more complex, tissue-specific segmentations typical of traditional neuroimaging workflows to test the models' ability to handle "near-raw" data.

Results and Discussion

The models were evaluated across six binary classification tasks using the Area Under the Curve (AUC) and accuracy metrics. The results indicate that 3D CNNs are highly effective at distinguishing healthy controls from patients with advanced Alzheimer's.

Task VoxCNN (AUC) ResNet (AUC)
AD vs. NC 0.88 0.87
AD vs. EMCI 0.66 0.67
LMCI vs. NC 0.67 0.65
EMCI vs. NC 0.57 0.58

The networks achieved approximately 80% accuracy in the AD vs. NC task. However, performance significantly decreased when classifying intermediate stages, such as EMCI vs. NC. This suggests that structural MRI alone may lack the sensitivity required to detect the subtle physical changes associated with early-stage cognitive impairment.

Automated Biomarker Discovery

A significant finding of this research is the models' ability to perform automated biomarker discovery. Using "occlusion sensitivity" analysis—where parts of the image are systematically blocked to observe the impact on prediction—the authors generated attention heatmaps.

Axial view of an occlusion sensitivity heatmap highlighting the hippocampus and ventricles.

The networks naturally focused on the hippocampus and ventricles, which are the primary anatomical regions used by clinicians to diagnose Alzheimer's. The fact that the models identified these regions without explicit anatomical labeling confirms that 3D CNNs can learn biologically meaningful features.

Interestingly, the performance of the plain VoxCNN was comparable to the deeper ResNet. This implies that for datasets of this scale (hundreds of samples), the benefits of extreme depth may be offset by the increased risk of overfitting, and a well-regularized plain CNN can be equally effective.

Conclusion

This research demonstrates that 3D CNNs are a viable alternative to traditional, labor-intensive neuroimaging pipelines. By achieving high accuracy in AD classification and autonomously identifying key anatomical biomarkers, these models pave the way for more automated diagnostic tools.

Future work should focus on addressing the "MCI challenge" by incorporating multi-modal data, such as PET scans or genetic markers, which may provide the additional information needed for early diagnosis. Furthermore, increasing sample sizes through data augmentation and multi-site datasets will be essential for scaling these deep learning approaches to clinical practice.

Summary updated

Abstract

In the recent years there have been a number of studies that applied deep learning algorithms to neuroimaging data. Pipelines used in those studies mostly require multiple processing steps for feature extraction, although modern advancements in deep learning for image classification can provide a powerful framework for automatic feature generation and more straightforward analysis. In this paper, we show how similar performance can be achieved skipping these feature extraction steps with the residual and plain 3D convolutional neural network architectures. We demonstrate the performance of the proposed approach for classification of Alzheimer's disease versus mild cognitive impairment and normal controls on the Alzheimers Disease National Initiative (ADNI) dataset of 3D structural MRI brain scans.
BibTeX
@inproceedings{korolev2017residual,
  author = {Sergey Korolev and Amir Safiullin and Mikhail Belyaev and Yulia Dodonova},
  title = {{Residual and plain convolutional neural networks for 3D brain MRI classification}},
  booktitle = {ISBI},
  year = {2017},
  url = {https://openreview.net/forum?id=oHvvAZOuaB},
}
Figure