Joint prediction of multiple scores captures better individual traits from brain images

Mehdi Rahim, Bertrand Thirion, Danilo Bzdok, Irène Buvat, Gaël Varoquaux

TL;DR

This paper introduces a "stacked multi-output learning" framework that jointly predicts multiple clinical scores from brain imaging data. This approach significantly improves prediction accuracy and robustness compared to traditional single-score prediction methods, offering a more nuanced understanding of individual brain-behavior relationships.

What Problem Does This Paper Address?

Understanding the complex relationship between brain structure/function and an individual's traits (behavioral, clinical) is a central goal in neuroscience. While neuroimaging techniques like fMRI can capture brain activity, relating these signals to rich descriptions of individuals (e.g., cognitive scores, disease status) is challenging. Existing methods often predict individual scores in isolation or rely on composite scores, potentially missing shared information across different traits. This can lead to models that are less accurate, less robust to noise, and less generalizable to new datasets. The paper aims to bridge this gap by developing a method that can jointly learn from and predict multiple clinical scores simultaneously, leveraging the inherent correlations between these scores to improve overall prediction.

Methods

The core innovation is a stacked multi-output learning framework. This framework operates in two main stages:

  1. Source-wise Multi-Output Prediction: For each input source (e.g., a connectivity map from a specific brain region), a multi-output model (specifically, a multi-task elastic-net regression) is trained to predict all clinical scores simultaneously. This encourages the model to learn shared patterns across scores within each source.
  2. Stacked Prediction: The predictions from all individual sources are then "stacked" together. This creates a new input matrix where each row represents a subject, and columns represent the predictions for each clinical score from each source. A second-level multi-output model (a random forest in this case) is then trained on this stacked input to make the final prediction of the clinical scores. This stacking approach allows the model to optimally combine information from different sources and learn complex, non-linear relationships.

This approach is contrasted with single-output models (predicting one score at a time), low-rank models like Canonical Correlation Analysis (CCA) and Reduced Rank Regression (RRR), and simpler aggregation strategies like majority voting and averaging.

Datasets

The study utilizes three publicly available datasets:

  • COBRE: A dataset focused on schizophrenia, containing rs-fMRI data and clinical scores.
  • ADNI: A large, multi-modal dataset for Alzheimer's Disease research, including rs-fMRI, anatomical MRI, and CSF biomarkers. This dataset was used for both training and testing, and also for cross-dataset validation.
  • ADNIDOD: A dataset from the US Department of Defense, focusing on brain aging and neurodegenerative diseases in Vietnam War veterans, used for cross-dataset validation with ADNI.

Key Insights and Ideas

  • Joint Prediction is Powerful: Predicting multiple related clinical scores together, rather than in isolation, unlocks shared information and leads to more accurate and robust predictions. This is akin to understanding a person holistically rather than focusing on isolated traits.
  • Exploiting Inter-Score Correlations: The framework explicitly leverages the fact that clinical scores are often correlated. By learning them jointly, the model can better disentangle signal from noise in individual scores.
  • Improved Generalizability: The stacked multi-output model shows better performance when transferring knowledge from one dataset to another (cross-dataset validation), suggesting it's less susceptible to dataset-specific idiosyncrasies.
  • Multi-Modal Fusion Benefits: The framework effectively integrates diverse data sources (fMRI, MRI, CSF) by treating predictions from each modality as a "source" to be stacked, outperforming simpler fusion methods.

Key Results

  • Improved Classification: For schizophrenia prediction on the COBRE dataset, the multi-output model achieved a significant increase in AUC (0.74 vs. 0.65 for single-output).
  • Improved Regression: For various clinical scores on both COBRE and ADNI datasets, the multi-output model consistently showed higher prediction accuracy (measured by $\Delta R^2$) compared to single-output models, with average gains of around 0.1.
  • Cross-Dataset Robustness: When training on ADNI and testing on ADNIDOD, the multi-output model significantly outperformed single-output models, demonstrating better generalization.
  • Superior to Low-Rank Models: The stacked multi-output model outperformed CCA and RRR in predicting clinical scores, with CCA often showing non-significant results.
  • Stacking Outperforms Aggregation: Stacking predictions from different sources using a random forest was more accurate and stable than simple averaging or majority voting for combining predictions.

Limitations and Future Work

  • Low SNR in rs-fMRI: The paper acknowledges the inherent low signal-to-noise ratio in rs-fMRI data, which can limit prediction accuracy, especially for certain cognitive scores.
  • Dataset Heterogeneity: While the model shows improved cross-dataset generalization, predicting across different studies remains challenging due to variability in sites, scanners, and protocols.
  • Future Directions: The authors suggest leveraging this framework for large multi-modal databases like UK Biobank to discover novel biomarkers and gain deeper insights into neuropathologies. They also highlight the potential for incorporating more complex predictors in the stacking stage and exploring more sophisticated multi-modal fusion strategies.

Summary updated

Abstract

Highlights • Subjects are described by multiple behavioral scores and diagnosis status. • Multi-output models capture shared information by predicting them jointly. • We validate our multi-output model extensively (multiple/across/multimodal datasets). • It improves prediction of each score, including patient-control classification. • Our stacking approach outperforms common multivariate analysis methods (CCA, RRR). Abstract To probe individual variations in brain organization, population imaging relates features of brain images to rich descriptions of the subjects such as genetic information or behavioral and clinical assessments. Capturing common trends across these measurements is important: they jointly characterize the disease status of patient groups. In particular, mapping imaging features to behavioral scores with predictive models opens the way toward more precise diagnosis. Here we propose to jointly predict all the dimensions (behavioral scores) that make up the individual profiles, using so-called multi-output models. This approach often boosts prediction accuracy by capturing latent shared information across scores. We demonstrate the efficiency of multi-output models on two independent resting-state fMRI datasets targeting different brain disorders (Alzheimer's Disease and schizophrenia). Furthermore, the model with joint prediction generalizes much better to a new cohort: a model learned on one study is more accurately transferred to an independent one. Finally, we show how multi-output models can easily be extended to multi-modal settings, combining heterogeneous data sources for a better overall accuracy. Graphical abstract Download : Download high-res image (313KB) Download : Download full-size image Previous article in issue Next article in issue Recommended articles Citing articles (8) View full text © 2017 Elsevier Inc. All rights reserved.
BibTeX
@article{rahim2017joint,
  author = {Mehdi Rahim and Bertrand Thirion and Danilo Bzdok and Irène Buvat and Gaël Varoquaux},
  title = {{Joint prediction of multiple scores captures better individual traits from brain images}},
  journal = {NeuroImage},
  year = {2017},
  doi = {10.1016/j.neuroimage.2017.06.072},
}
Figure