Improved Brain Age Estimation with Slice-based Set Networks

Umang Gupta, Pradeep Lam, Greg Ver Steeg, Paul Thompson

Abstract

Deep Learning for neuroimaging data is a promising but challenging direction. The high dimensionality of 3D MRI scans makes this endeavor compute and data-intensive. Most conventional 3D neuroimaging methods use 3D-CNN-based architectures with a large number of parameters and require more time and data to train. Recently, 2D-slice-based models have received increasing attention as they have fewer parameters and may require fewer samples to achieve comparable performance. In this paper, we propose a new architecture for BrainAGE prediction. The proposed architecture works by encoding each 2D slice in an MRI with a deep 2D-CNN model. Next, it combines the information from these 2D-slice encodings using set networks or permutation invariant layers. Experiments on the BrainAGE prediction problem, using the UK Biobank dataset, showed that the model with the permutation invariant layers trains faster and provides better predictions compared to other state-of-the-art approaches.

Introduction

Predicting "brain age"—the biological age of a brain as seen on an MRI—is a powerful biomarker. A "Brain Age Gap" (the difference between chronological age and predicted brain age) can signal risks for neurological diseases like Alzheimer's.

However, training models to do this is difficult. Standard 3D Convolutional Neural Networks (3D-CNNs) are "parameter-hungry" and require massive amounts of data that medical fields often lack. Previous attempts to use 2D slices (which are easier to process) relied on Recurrent Neural Networks (RNNs/LSTMs) to process slices in a specific order (e.g., front-to-back). This is problematic because the "optimal" order is unknown, and RNNs are notoriously slow to train because gradients must flow through the entire sequence.

Methods

The authors propose a Slice-based Set Network. The architecture follows three main steps:

  1. 2D Encoding: Each 2D slice of a 3D MRI is passed through a shared 2D-CNN encoder to create a compact mathematical representation (embedding).
  2. Permutation Invariant Aggregation: Instead of treating slices as a sequence, the model treats them as a set. It uses operations that don't care about order, such as mean/max pooling or attention mechanisms.
  3. Prediction: The aggregated data is passed through a simple feed-forward network to predict the final age.
The aggregation block of the proposed Slice-based Set Network architecture.

By using permutation invariant layers, the model avoids the "vanishing gradient" problems of RNNs and the massive parameter count of 3D-CNNs.

Experimental Setup

The study utilized the UK Biobank dataset, specifically a subset of 10,446 healthy subjects (no psychiatric diagnoses).

  • Input: Structural MRI scans downsampled to 91 × 109 × 91.
  • Split: 7,312 for training, 2,194 for validation, and 940 for testing.
  • Mean Age: 62.6 years (Standard Deviation: 7.4 years).

Key Results and Insights

Training Efficiency and Convergence

The most significant finding is that the spatial sequence of slices isn't strictly necessary for accurate age estimation. Treating slices as a set simplifies the learning task and speeds up convergence. As shown in the training curves, the set-based models start learning almost immediately, whereas the 2D-slice-RNN baseline shows a "flat" loss curve for the first 20 epochs.

MAE on validation set during training, comparing the proposed attention-based model against 3D-CNN and RNN baselines.

Performance and Model Size

  • Accuracy: The 2D-slice-mean model achieved the lowest Mean Absolute Error (MAE) of 2.823 years, outperforming the 3D-CNN (3.017) and the previous state-of-the-art 2D-slice-RNN (3.002).
  • Model Size: The proposed models used roughly 1 million parameters, compared to nearly 3 million for the 3D-CNN, making them much more computationally efficient.
  • Extreme Robustness: Because the model treats slices as a set, it is remarkably tolerant to data loss. When 80% of the slices were randomly removed, the 2D-slice-mean model still maintained an MAE of 3.35 years, whereas RNN-based models degraded significantly.

Conclusion

Slice-based set networks provide a more efficient and robust alternative to traditional 3D-CNNs for brain age estimation. Interestingly, the simple 2D-slice-mean model performed slightly better than the more complex 2D-slice-attention model, suggesting that age-related information is distributed fairly evenly across the brain. Future work will focus on validating these models on "out-of-distribution" samples, such as patients with neurodegenerative diseases, and exploring transfer learning from non-medical datasets.

Summary updated

Abstract

Deep Learning for neuroimaging data is a promising but challenging direction. The high dimensionality of 3D MRI scans makes this endeavor compute and data-intensive. Most conventional 3D neuroimaging methods use 3D-CNN-based architectures with a large number of parameters and require more time and data to train. Recently, 2D-slice-based models have received increasing attention as they have fewer parameters and may require fewer samples to achieve comparable performance. In this paper, we propose a new architecture for BrainAGE prediction. The proposed architecture works by encoding each 2D slice in an MRI with a deep 2D-CNN model. Next, it combines the information from these 2D-slice encodings using set networks or permutation invariant layers. Experiments on the BrainAGE prediction problem, using the UK Biobank dataset, showed that the model with the permutation invariant layers trains faster and provides better predictions compared to other state-of-the-art approaches.
BibTeX
@inproceedings{gupta2021improved,
  author = {Umang Gupta and Pradeep Lam and Greg Ver Steeg and Paul Thompson},
  title = {{Improved Brain Age Estimation with Slice-based Set Networks}},
  booktitle = {ISBI},
  year = {2021},
  doi = {10.1109/isbi48211.2021.9434081},
}
Figure