REPRODUCIBILITY OF BRAIN AGE SALIENCIES ACROSS DEEP NEURAL NETWORK ARCHITECTURES

Nicholas Kim, Phoebe Imms, Nikhil N. Chaudhari, Nahian F. Chowdhury, Anar Amgalan, Paul Bogdan, Andrei IrimiaInnovation 2023 · DOI

TL;DR

This paper investigates how different Convolutional Neural Network (CNN) architectures affect the interpretability of brain age estimation, specifically focusing on sex differences in brain aging. The study found that while specific brain regions highlighted by CNN saliency maps varied slightly between architectures, the overall patterns of sex-based aging differences remained consistent, suggesting robust capture of aging dynamics.

What Problem Does This Paper Address?

Estimating an individual's "biological age" (BA) from brain scans (like MRIs) is a growing area of research, aiming to capture age-related structural changes beyond chronological age. Convolutional Neural Networks (CNNs) are powerful tools for this, and importantly, they can provide "saliency maps." These maps highlight which parts of the brain are most influential in the CNN's BA prediction, offering a window into how the network is making its age estimate. Previous work has suggested that brain aging might differ between sexes, with different brain regions showing more pronounced changes. However, a key question is whether the interpretability tools (saliency maps) themselves are reliable and reproducible across different CNN models. If different CNN architectures highlight different brain regions as important for BA estimation, it could undermine our confidence in understanding sex differences in brain aging. This paper tackles the reproducibility of these saliency maps and the insights they provide regarding sex differences in brain aging.

Methods

The researchers employed two distinct CNN architectures for brain age estimation using T1-weighted MRI scans. Both CNNs were designed for regression tasks, with the MRI data as input and the estimated biological age as output. The core innovation lies in comparing the interpretability of these models.

  • CNN Architecture 1: Trained on male and female participants separately.
  • CNN Architecture 2: Trained on both male and female participants together.

The interpretability was assessed using saliency maps, which quantify the importance of different brain regions for the BA prediction. These maps were then compared to understand the reproducibility of findings, particularly concerning sex differences in brain aging. Simulated alterations in brain morphometry were used to validate that the saliency maps were indeed highlighting regions with aging-related dynamics.

Datasets

The study utilized a substantial dataset comprising T1-weighted MRIs from 5,851 cognitively normal individuals, with 3,142 being female. The participants ranged in age from 22 to 95 years. The data was aggregated from several well-established neuroimaging initiatives:

  • Alzheimer's Disease Neuroimaging Initiative (ADNI) (N = 510)
  • Human Connectome Project Aging (HCP-Aging) (N = 508)
  • Human Connectome Project Young Adult (HCP-YA) (N = 1,112)
  • UK Biobank (N = 3,721)

This multi-site, large-scale dataset provides a robust foundation for training and validating the CNN models and their interpretability.

Key Insights and Ideas

  • Robustness of Sex Differences: Despite using different CNN architectures (one trained separately by sex, one jointly), the study found that the overall patterns of sex differences in brain aging were consistently identified. This suggests that the underlying biological differences in how brains age between males and females are robustly captured by these models.
  • Saliency Maps as Reliable Indicators: The saliency maps, even when generated by different CNN architectures, proved effective in highlighting brain regions that exhibit age-related changes. This validates the use of saliency maps as a tool to understand the "why" behind brain age predictions.
  • Architectural Differences Don't Derail Core Findings: While there were some variations in the specific brain regions emphasized by each CNN architecture, these differences did not fundamentally alter the conclusions about sex-specific aging patterns. This is a crucial finding for the field, indicating a degree of model-agnosticism in the interpretation of brain aging.

Key Results

The study revealed significant sex differences in brain regions contributing to biological age estimation:

  • Males: BA estimation relied significantly more on the right lateral temporal lobe and superior frontal gyrus (p < 0.05).
  • Females: BA estimation depended significantly more on the right posterior and bilateral occipital regions, and the medial parietal lobe (p < 0.05).

These findings were consistent across both CNN architectures when compared to a null distribution. Simulated morphometric changes confirmed that the saliency maps accurately reflected aging dynamics in these identified regions.

Limitations and Future Work

  • Limitations: The paper acknowledges that variations in CNN model saliencies did not affect overall anatomic patterns, but it implies there were variations. The authors suggest that novel statistical models are needed to formally compare CNN saliencies, especially given their nonlinear nature. This indicates that while the broad conclusions are reproducible, a deeper, more nuanced comparison of saliency maps between models is still an open challenge.
  • Future Work: The authors implicitly suggest the need for advanced statistical methods to rigorously compare and quantify differences in saliency maps between various CNN architectures. Developing such methods would allow for a more precise understanding of how architectural choices influence interpretability and could lead to more standardized approaches for analyzing brain aging.

Summary updated

Abstract

Abstract The brain’s biological age (BA) reflects structural changes related to neuroanatomic senescence. BA is estimated from magnetic resonance images (MRIs) using convolutional neural networks (CNNs), which provide interpretability through saliency maps. By conveying the importance of brain regions to BA estimation, these maps reveal sex differences in brain-aging. We compared two CNN architectures to assess the reproducibility of such differences. Both regression CNNs had T1-weighted MRIs as inputs; outputs were estimated BAs. One CNN was trained on males and females separately, whereas the other was trained on both sexes. The dataset used consisted of T1-weighted MRIs from 5,851 cognitively normal individuals (3,142 females) aged 22 to 95 years, sampled from the Alzheimer’s Disease Neuroimaging Initiative (N = 510), Human Connectome Project Aging (N = 508) and Young Adult (N = 1,112), and UK Biobank (N = 3,721). For both models, compared to the average saliency for a null distribution, males’ BA estimation relied significantly (p &amp;lt; 0.05) more on the right lateral temporal lobe and superior frontal gyrus. Females’ BA estimation depended significantly (p &amp;lt;; 0.05) more on the right posterior and bilateral occipital regions, and medial parietal lobe. Simulated alterations in brain morphometry indicated saliencies correctly revealed regions with aging-related dynamics, which confirm prior findings on sex differences in brain-aging. Variations in CNN models’ saliencies did not affect overall anatomic patterns, suggesting that CNNs can capture brain-aging patterns robustly despite architecture differences. Novel statistical models for formal comparison of CNN saliencies should be developed to accommodate their nonlinear nature.
BibTeX
@inproceedings{kim2023reproducibility,
  author = {Nicholas Kim and Phoebe Imms and Nikhil N. Chaudhari and Nahian F. Chowdhury and Anar Amgalan and Paul Bogdan and Andrei Irimia},
  title = {{REPRODUCIBILITY OF BRAIN AGE SALIENCIES ACROSS DEEP NEURAL NETWORK ARCHITECTURES}},
  booktitle = {Innovation},
  year = {2023},
  doi = {10.1093/geroni/igad104.3572},
}
Figure