Abstract
“Predicted brain age” refers to a biomarker of structural brain health derived from machine learning analysis of T1-weighted brain magnetic resonance (MR) images. A range of machine learning methods have been used to predict brain age, with convolutional neural networks (CNNs) currently yielding state-of-the-art accuracies. Recent advances in deep learning have introduced transformers, which are conceptually distinct from CNNs, and appear to set new benchmarks in various domains of computer vision. However, transformers have not yet been applied to brain age prediction. Thus, we address two research questions: First, are transformers superior to CNNs in predicting brain age? Second, do conceptually different deep learning model architectures learn similar or different “concepts of brain age”? We adapted a Simple Vision Transformer (sViT) and a Shifted Window Transformer (SwinT) to predict brain age, and compared both models with a ResNet50 on 46,381 T1-weighted structural MR images from the UK Biobank. We found that SwinT and ResNet performed on par, while additional training samples will most likely give SwinT the edge in prediction accuracy. We identified that different model architectures may characterize different (sub-)sets of brain aging effects, representing diverging concepts of brain age. Thus, we systematically tested whether sViT, SwinT and ResNet focus on different concepts of brain age by examining variations in their predictions and clinical utility for indicating deviations in neurological and psychiatric disorders. Reassuringly, we did not find substantial differences in the structure of brain age predictions between model architectures. Based on our results, the choice of deep learning model architecture does not appear to have a confounding effect on brain age studies.
Introduction
"Brain age" is a popular biomarker calculated by comparing a person’s chronological age to the age predicted by a machine learning model analyzing their brain MRI. A "gap" between the two—the Brain Age Gap (BAG)—can indicate neurodegenerative diseases or poor lifestyle factors.
Currently, Convolutional Neural Networks (CNNs) are the workhorse of this field. However, the computer vision world is rapidly shifting toward Transformers. Unlike CNNs, which focus on local patterns (like textures in a specific brain region), Transformers use "attention" mechanisms to weigh global relationships across the entire image. This study addresses a critical concern: If we change the mathematical architecture under the hood, do we change the clinical result? If a Transformer "sees" different aging effects than a CNN, then different studies might reach conflicting conclusions about a disease simply because they used different model architectures.
Methodology
The researchers adapted and compared three distinct 3D deep learning architectures using a massive dataset from the UK Biobank (UKBB), totaling 46,381 T1-weighted structural MRIs. The models included:
- 3D ResNet50: The industry-standard CNN baseline.
- 3D Simple Vision Transformer (sViT): A basic transformer that treats the brain as a sequence of 3D cubes.
- 3D Shifted Window Transformer (SwinT): A more sophisticated transformer that reintroduces hierarchical properties to handle large images efficiently.
The study design focused on both prediction accuracy and clinical utility. By analyzing how sensitive each model's BAG was to various neurological disorders and lifestyle factors, the team could determine if the models converged on the same biological "concept" of aging.
Results
Prediction Accuracy and Scaling
ResNet50 and SwinT were neck-and-neck in performance, with Mean Absolute Errors (MAE) of 2.66 years and 2.67 years, respectively. While they performed similarly on the current dataset, power-law extrapolations suggest that SwinT’s accuracy improves more sharply as more data is added. The authors predict SwinT will become the definitive choice once training sets consistently exceed ~25,000 samples, as ResNet’s performance begins to plateau.
Clinical Consistency
Despite their fundamental mathematical differences, the models converged on the same "concept" of brain age. The predictions between a CNN and a Transformer were highly correlated ($r = 0.94$). Crucially, the BAG remained a consistent biomarker across architectures. Effect sizes (Cohen’s $d$) for identifying diseases were nearly identical across models. For instance, all models identified Multiple Sclerosis as having the most significant impact on brain aging, followed by Parkinson's and Alcohol Use Disorder.
Discussion and Conclusion
The findings suggest that the structural features of aging are robust enough that they are not "hidden" from specific types of neural networks. The "concept" of brain age learned by the models was stable across different random initializations and training runs, particularly for the SwinT.
However, several limitations remain. Transformers are computationally intensive, limiting exhaustive hyperparameter optimization for 3D volumes. Furthermore, while the models learn the same concept, future work is needed to explicitly map out what that concept is in 3D space using explainable AI (XAI) methods. Finally, as the UK Biobank represents a specific demographic, further validation across more diverse global datasets is necessary to ensure these findings are universal.
In conclusion, the choice of deep learning model architecture—whether CNN or Transformer—does not appear to have a confounding effect on brain age studies, ensuring that clinical findings remain consistent as the field adopts newer technologies.
Summary updated