Translating phenotypic prediction models from big to small anatomical MRI data using meta-matching

Naren Wulan, Lijun An, Chen Zhang, Ru Kong, Pansheng Chen, Danilo Bzdok, Simon B. Eickhoff, Avram J. Holmes, B.t. Thomas Yeo

TL;DR

This paper introduces "meta-matching," a framework that effectively translates brain MRI-based prediction models trained on large datasets to small datasets, significantly outperforming traditional methods like elastic net and transfer learning. This advancement is crucial for making accurate phenotypic predictions in neuroscience research, where small sample sizes are common.

What Problem Does This Paper Address?

In neuroscience, a major goal is to predict individual traits (phenotypes) from brain structure, particularly using T1-weighted MRI. While prediction accuracy generally improves with larger datasets, many research questions, especially those involving specific clinical populations, are limited to small sample sizes (fewer than 200 participants). This scarcity of data hinders the development of reliable predictive models. The authors aim to bridge this gap by developing a method to leverage knowledge from large, publicly available datasets to improve predictions on these smaller, often unavoidable, datasets.

Methods

The core of this work is the adaptation and application of two "meta-matching" strategies: "meta-matching finetune" and "meta-matching stacking." These methods build upon previous work that used resting-state functional connectivity, but here they are adapted for T1-weighted MRI data. The key innovation is the use of a 3D convolutional neural network (CNN) as the base model.

  • Meta-matching Finetune: This approach involves training a 3D CNN on a large dataset (meta-training set) to predict multiple phenotypes. For a new, small dataset (meta-test set), the last layers of this pre-trained CNN are "finetuned." Crucially, the initialization of these last layers is not random but is guided by selecting the "closest" phenotypic prediction model from the meta-training set, leveraging learned relationships between phenotypes.
  • Meta-matching Stacking: This method also uses a pre-trained 3D CNN. However, instead of finetuning the last layers, it selects the top-performing predictions from the CNN for a given phenotype and uses these predictions as input features for a secondary model (kernel ridge regression). This "stacks" the predictions from the CNN to build a more robust final prediction.

These meta-matching approaches are compared against two baselines:

  • Elastic Net: A standard linear regression model using pre-defined morphometric features (thickness and volume) derived from T1 MRI.
  • Classical Transfer Learning: This involves training a 3D CNN on the meta-training set and then randomly initializing and finetuning the last layer on the meta-test set.

Datasets

The study utilizes three large-scale, publicly available neuroimaging datasets:

  • UK Biobank (UKBB): A massive dataset with over 500,000 participants. For this study, 36,461 participants with T1 structural MRI and 67 non-brain-imaging phenotypes were used. This served as the primary large-scale source dataset.
  • Human Connectome Project Young Adults (HCP-YA): A dataset of 1,017 healthy young adults, with 35 non-brain-imaging phenotypes.
  • Human Connectome Project Aging (HCP-Aging): A dataset of 656 healthy older adults, with 45 non-brain-imaging phenotypes.

These datasets differ in age ranges, scanners, acquisition protocols, and demographics, providing a rigorous test for the generalizability of the meta-matching framework.

Key Insights and Ideas

  • Leveraging Phenotype Correlations is Key: The success of meta-matching hinges on the idea that phenotypes are correlated. By exploiting these correlations, models trained to predict one phenotype can provide valuable insights for predicting a related, but unseen, phenotype in a smaller dataset.
  • T1 MRI is Sufficient for Meta-Matching: While previous meta-matching work focused on functional connectivity, this study demonstrates its effectiveness with simpler T1-weighted anatomical MRI data, making it more broadly applicable.
  • Meta-Matching Outperforms Standard Methods: The results consistently show that meta-matching significantly outperforms both elastic net and classical transfer learning, especially when dealing with very small sample sizes (K < 200).
  • Different Meta-Matching Variants Excel on Different Metrics: Meta-matching stacking performed best for Pearson's correlation (a measure of relative prediction performance), while meta-matching finetune was superior for Coefficient of Determination (COD, a measure of absolute prediction performance).
  • Interpretability is Possible: The study shows that meta-matching models can be interpreted using techniques like the Haufe transform, revealing which brain regions are important for predicting specific phenotypes and offering insights into the underlying biology.

Key Results

  • Within-Dataset Performance: In the UK Biobank, both meta-matching finetune and stacking significantly outperformed elastic net and classical transfer learning across various sample sizes (K).
  • Cross-Dataset Performance: When translating models from the UK Biobank to the HCP-YA and HCP-Aging datasets (with different scanners, demographics, etc.), meta-matching still showed substantial improvements. For example, meta-matching finetune achieved a 136% improvement in variance explained over transfer learning when translating a UK Biobank model to 100 HCP-YA participants.
  • Small Sample Size Challenge: For very small sample sizes (K ≤ 20), absolute prediction accuracy (COD) was generally negative or near zero for all methods, indicating that reliable absolute predictions are difficult to achieve without sufficient data.
  • Phenotype-Specific Improvements: The benefits of meta-matching were not uniform across all phenotypes. Phenotypes with stronger correlations to those in the meta-training set showed greater improvements.

Limitations and Future Work

  • Phenotype Correlation Dependency: The effectiveness of meta-matching is dependent on the correlation between phenotypes in the meta-training and meta-test sets. If these correlations are weak, the improvement may be limited. This limitation is acknowledged as inherent to most meta-learning and transfer learning approaches.
  • Computational Cost: The bootstrapping procedure for statistical testing was computationally expensive, precluding its use for comparing meta-matching finetune and classical transfer learning in some cases.
  • Future Work: The authors propose aggregating more diverse large-scale datasets, targeting different populations (children, lifespan, various disorders), and including a wider range of phenotypic measures and MRI acquisition parameters. This will allow for more robust pretraining and potentially further improvements in meta-matching approaches.

Summary updated

Abstract

Individualized phenotypic prediction based on structural MRI is an important goal in neuroscience. Prediction performance increases with larger samples, but small-scale datasets with fewer than 200 participants are often unavoidable. We have previously proposed a "meta-matching" framework to translate models trained from large datasets to improve the prediction of new unseen phenotypes in small collection efforts. Meta-matching exploits correlations between phenotypes, yielding large improvement over classical machine learning when applied to prediction models using resting-state functional connectivity as input features. Here, we adapt the two best performing meta-matching variants ("meta-matching finetune" and "meta-matching stacking") from our previous study to work with T1-weighted MRI data by changing the base neural network architecture to a 3D convolution neural network. We compare the two meta-matching variants with elastic net and classical transfer learning using the UK Biobank (N = 36,461), Human Connectome Project Young Adults (HCP-YA) dataset (N = 1,017) and HCP-Aging dataset (N = 656). We find that meta-matching outperforms elastic net and classical transfer learning by a large margin, both when translating models within the same dataset, as well as translating models across datasets with different MRI scanners, acquisition protocols and demographics. For example, when translating a UK Biobank model to 100 HCP-YA participants, meta-matching finetune yielded a 136% improvement in variance explained over transfer learning, with an average absolute gain of 2.6% (minimum = -0.9%, maximum = 17.6%) across 35 phenotypes. Overall, our results highlight the versatility of the meta-matching framework.
BibTeX
@inproceedings{wulan2024translating,
  author = {Naren Wulan and Lijun An and Chen Zhang and Ru Kong and Pansheng Chen and Danilo Bzdok and Simon B. Eickhoff and Avram J. Holmes and B.T. Thomas Yeo},
  title = {{Translating phenotypic prediction models from big to small anatomical MRI data using meta-matching}},
  booktitle = {Imaging},
  year = {2024},
  doi = {10.1162/imag\_a\_00251},
}
Figure