Beyond functional connectivity: deep learning applied to resting-state fMRI time series in the prediction of 58 human traits in the HCP

Bruno Hebling Vieira, Mikkel Schöttner, Vince D. Calhoun, Carlos Ernesto Garrido SalmónbioRxiv 2024 · DOI

TL;DR

This paper explores whether deep learning can improve the prediction of human traits from fMRI data by directly learning from raw time series, rather than relying on pre-computed functional connectivity. The study finds that while deep learning (specifically InceptionTime) and traditional methods (Kernel Ridge Regression) perform comparably on most traits, deep learning shows advantages in robustness to smaller training datasets and offers a more nuanced view of neuroimaging-based behavior prediction.

What Problem Does This Paper Address?

Predicting individual human traits from brain imaging data is a growing area of research, aiming to understand the complex relationship between brain structure/function and behavior. Traditionally, this has involved extracting features like functional connectivity (how different brain regions communicate) and then using machine learning models to predict traits. However, the effectiveness of deep learning (DL) in this domain, especially when applied to raw fMRI time series, has yielded mixed results. The authors hypothesize that relying on pre-computed functional connectivity might be a bottleneck, preventing DL models from fully leveraging their ability to learn complex, hierarchical representations directly from the data. This paper aims to investigate whether bypassing this feature extraction step and feeding raw fMRI time series into a DL model can lead to improved prediction accuracy compared to traditional methods.

Methods

The core of this study involves comparing two main approaches for predicting 58 behavioral traits from resting-state fMRI (RS-fMRI) data:

  • Deep Learning (TCNN): The authors adapted the InceptionTime architecture, a 1D convolutional neural network (CNN) specifically designed for time series classification. This model directly processes regional fMRI time series, learning complex representations without manual feature engineering. Key modifications included using leaky ReLU activations and a specific learning rate decay schedule.
  • Traditional Machine Learning (KRR): Kernel Ridge Regression (KRR) was used as a strong baseline. This method utilized pre-computed resting-state functional connectivity (RSFC) matrices derived from the fMRI data. The correlation coefficient between the upper-triangular part of the average RSFC matrix was used as the kernel.

Both models were trained and evaluated using a 20-fold cross-validation strategy. The study also investigated the impact of covariate regression (controlling for factors like age, sex, etc.) on performance, both during training and as a post-hoc analysis.

Datasets

The primary dataset used in this study is the Human Connectome Project (HCP) dataset. This dataset is known for its large sample size and rich collection of neuroimaging and behavioral data. The authors specifically utilized the RS-fMRI data and 58 behavioral measures from the HCP. No new benchmarks were introduced; rather, the study leverages existing, well-characterized data to compare established and novel modeling approaches.

Key Insights and Ideas

  • Direct Time Series Learning: The study demonstrates that deep learning models can learn meaningful predictive features directly from raw fMRI time series, potentially overcoming limitations of pre-computed functional connectivity.
  • Nuanced Performance Comparison: While both DL and KRR achieve comparable performance on most traits, significant differences emerge when considering specific metrics (MSE vs. correlation) and the impact of covariates.
  • Robustness to Data Size: The TCNN approach showed greater robustness to reductions in training dataset size compared to KRR, suggesting it can generalize better with less data.
  • Time Series Length Sensitivity: Conversely, the TCNN was more sensitive to the length of the time series at test time, highlighting a trade-off in its performance characteristics.
  • Importance of Performance Metric: The choice of performance metric (Pearson's correlation vs. MSE) significantly impacts the observed differences between models, underscoring the need for careful consideration in neuroimaging research.

Key Results

  • Both TCNN and KRR models could predict a substantial number of traits (41 and 42 respectively) with significant correlation coefficients, with 40 traits in common.
  • When using Mean Squared Error (MSE) as the performance metric, significant differences were observed in scaling. The TCNN achieved lower MSE in 7 out of 11 significant differences without covariate regression, and this number increased to 11 out of 13 when accounting for covariates.
  • The TCNN was more robust to reductions in training set size, showing less degradation in performance compared to KRR.
  • The TCNN's performance was more affected by reductions in time series length at test time, while KRR showed less sensitivity.
  • For grip strength, the TCNN predicted the 58 traits jointly, which may explain its better performance when using smaller training sets.

Limitations and Future Work

  • Data Leakage Risk: The authors acknowledge a potential risk of optimistic results for KRR due to the use of information from the whole dataset for comparison, although they minimized this by following established procedures.
  • Selection Bias: Excluding subjects with incomplete runs or excessive head motion might introduce selection bias, as these subjects could systematically differ from those included.
  • Generalizability: The study notes that models trained on specific demographics (e.g., white Americans) may not generalize to others, highlighting the need for further research on diverse populations.
  • Confounder Handling: The authors opted for a post-hoc covariate regression strategy, acknowledging that more integrated approaches might be beneficial.
  • Future Work: The authors suggest exploring larger datasets, incorporating other fMRI tasks, and investigating more advanced methods for handling confounders to improve generalizability and prediction accuracy. They also emphasize the need for further research into the neurobiological basis of behavior and the role of temporal dynamics in BOLD fluctuations.

Summary updated

Abstract

Abstract Machine learning has made several inroads into the study of brain-behavior relations based on in vivo imaging. While the advent of deep neural networks was expected to further improve predictions, the current literature based on resting-state functional connectivity presents mixed results. We hypothesize that the representation of the data, i.e. in the form of functional connectivity, could restrict an advantage of deep learning techniques, namely that of learning complex representations directly from the data. Thus, we investigated if bypassing this feature extraction resulted in improved performance in the prediction of 58 widely studied behavioral traits from a large sample of Human Connectome Project subjects, using deep learning techniques. For this task, we adapted the InceptionTime architecture, which jointly predicts traits directly from regional time series through representation learning, and compared results with a strong kernel-based baseline. Results revealed that both models achieve comparable performance in most traits. Eleven significant differences in mean squared error were detected, however, with seven favoring the neural network approach, and this number increased when accounting for covariates. We additionally show that contrary to the expectation, the neural network approach was more robust to reductions in the training set size. On the other hand, it was more sensitive to reductions in the length of the time series at test time. Our results present a more nuanced view of the potential of deep learning for the prediction of behavior from neuroimaging, which allows learning features directly from the data.
BibTeX
@misc{vieira2024beyond,
  author = {Bruno Hebling Vieira and Mikkel Schöttner and Vince D. Calhoun and Carlos Ernesto Garrido Salmón},
  title = {{Beyond functional connectivity: deep learning applied to resting-state fMRI time series in the prediction of 58 human traits in the HCP}},
  year = {2024},
  doi = {10.1101/2024.03.07.583858},
  howpublished = {bioRxiv},
}
Figure