Representational learning of brain responses in executive function and higher-order cognition using deep graph convolutions

Yu Zhang, Nicolas Farrugia, Alain Dagher, Pierre BellecbioRxiv 2021 · DOI

TL;DR

This paper introduces ChebNet, a deep graph convolutional network, to decode complex human cognition from brain activity. By modeling both localized brain activity (segregation) and interactions across brain networks (integration), ChebNet achieves high accuracy in distinguishing cognitive states, particularly for higher-order functions, and reveals insights into the genetic basis of brain-behavior relationships.

What Problem Does This Paper Address?

Decoding cognitive states from neuroimaging data (like fMRI) has been a long-standing goal in neuroscience. Traditional methods often focus on specific brain regions or simple cognitive tasks (like motor or sensory processes), achieving good results but struggling with the complexity of higher-order cognition (e.g., working memory, executive functions). This limitation stems from the computational challenges of analyzing whole-brain data and the tendency to overlook how different brain regions dynamically interact. The paper aims to bridge this gap by developing a more sophisticated decoding model that can capture both localized brain activity and the intricate functional integration across the entire brain, enabling a more comprehensive understanding of human cognition.

Methods

The core of this work is the ChebNet graph convolutional network. Unlike simpler graph convolutional networks, ChebNet uses a truncated expansion of Chebyshev polynomials to approximate graph convolutions. This allows it to capture information integration at multiple scales:

  • K=0: Global scaling (akin to traditional univariate analysis).
  • K=1: Integration within immediate neighbors (e.g., within a brain network).
  • K>1: Integration across larger neighborhoods, including interactions between different brain networks and systems.

The authors construct a brain graph where nodes represent brain regions (using the Glasser's multi-modal parcellation of 360 areas) and edges represent functional connectivity derived from resting-state fMRI. Task-evoked fMRI data is then mapped onto this graph, and ChebNet layers learn hierarchical representations of this spatiotemporal brain activity. The model's performance is evaluated by its ability to decode cognitive states and by analyzing the learned representations using techniques like Centered Kernel Alignment (CKA) and t-SNE.

Datasets

The study primarily uses the Human Connectome Project (HCP) S1200 release task-fMRI dataset. This dataset includes fMRI data from over 1000 subjects performing various cognitive tasks across six domains: Working Memory, Motor, Language, Social Cognition, Relational Processing, and Emotion. The data was preprocessed using standard HCP pipelines.

Key Insights and Ideas

  • Functional Integration is Crucial for High-Order Cognition: The study strongly suggests that decoding complex cognitive functions, especially those beyond basic motor and sensory processes, heavily relies on understanding how different brain networks integrate information. Simple segregation of localized activity is insufficient.
  • ChebNet Captures Hierarchical Brain Organization: The ChebNet architecture naturally learns hierarchical representations, mirroring the known hierarchical organization of brain function. Different layers capture different levels of information integration, from local to global.
  • K-order Matters: The "K-order" of ChebNet (which dictates the scale of information integration) significantly impacts decoding performance. Higher K-orders are essential for tasks requiring complex inter-network communication, while simpler tasks might benefit from lower K-orders.
  • Brain Decoding Reveals Biological Basis: Saliency maps derived from the ChebNet model highlight brain regions that are critical for decoding specific cognitive states. These regions often align with known functional areas, suggesting that the model learns biologically meaningful features.
  • Genetic Basis of Brain-Behavior Links: The research provides evidence that individual variability in ChebNet representations of working memory tasks is heritable and strongly associated with behavioral performance, suggesting shared genetic influences on both brain activity patterns and cognitive abilities.

Key Results

  • High Decoding Accuracy: The ChebNet model achieved an average test accuracy of 96% in distinguishing six cognitive domains using only 10 seconds of fMRI data. It could also distinguish fine-grained cognitive states on a trial basis with over 93% accuracy.
  • Superior Performance: ChebNet outperformed existing linear and nonlinear decoding models, including other deep learning architectures, on the HCP dataset.
  • Task-Specific Integration Needs: For motor tasks, a simpler ChebNet model (K=1) was sufficient, indicating reliance on localized activity and within-network integration. For working memory tasks, higher K-orders (up to K=5) were necessary, highlighting the importance of between-network integration.
  • Behavioral Associations: ChebNet representations showed significantly stronger associations with in-scanner task performance (e.g., accuracy and reaction time) compared to raw fMRI data.
  • Heritability: Significant heritability estimates were found for ChebNet graph representations and WM task performance, with evidence of shared genetic influences.

Limitations and Future Work

  • Computational Complexity: While ChebNet offers advantages, the computational demands of graph convolutions, especially with high K-orders and large brain graphs, can be significant.
  • Generalizability: While the model showed good generalizability across cognitive domains, further validation on diverse datasets and tasks would be beneficial.
  • Future Directions: The authors suggest exploring more naturalistic tasks (like watching movies or playing video games) and further investigating the genetic underpinnings of brain-behavior relationships using their framework. They also highlight the potential for ChebNet to uncover more nuanced organizational principles in brain cognition.

Summary updated

Abstract

Abstract Brain decoding aims to infer human cognition from recordings of neural activity using modern neuroimaging techniques. Studies so far often concentrated on a limited number of cognitive states and aimed to classifying patterns of brain activity within a local area. This procedure demonstrated a great success on classifying motor and sensory processes but showed limited power over higher cognitive functions. In this work, we investigate a high-order graph convolution model, named ChebNet, to model the segregation and integration organizational principles in neural dynamics, and to decode brain activity across a large number of cognitive domains. By leveraging our prior knowledge on brain organization using a graph-based model, ChebNet graph convolution learns a new representation from task-evoked neural activity, which demonstrates a highly predictive signature of cognitive states and task performance. Our results reveal that between-network integration significantly boosts the decoding of high-order cognition such as visual working memory tasks, while the segregation of localized brain activity is sufficient to classify motor and sensory processes. Using twin and family data from the Human Connectome Project (n = 1,070), we provide evidence that individual variability in the graph representations of working-memory tasks are under genetic control and strongly associated with participants in-scanner behaviors. These findings uncover the essential role of functional integration in brain decoding, especially when decoding high-order cognition other than sensory and motor functions. Teaser Modelling functional integration through graph convolution is a necessary step towards decoding high-order human cognition. Significance statement Over the past two decades, many studies have applied multivariate pattern analysis to decode what task a human participant is performing, based on a scan of her brain. The vast majority of these studies have however concentrated on select regions and a specific domain, because of the computational complexity of handling full brain data in a multivariate model. With the fast progress in the field of deep learning, it is now possible to decode a variety of cognitive domains simultaneously using a full-brain model. By leveraging our prior knowledge on brain organization using a graph-based model, we uncovered different organizational principles in brain decoding for motor execution and high-order cognition by modelling functional integration through graph convolution.
BibTeX
@misc{zhang2021representational,
  author = {Yu Zhang and Nicolas Farrugia and Alain Dagher and Pierre Bellec},
  title = {{Representational learning of brain responses in executive function and higher-order cognition using deep graph convolutions}},
  year = {2021},
  doi = {10.1101/2021.07.26.453914},
  howpublished = {bioRxiv},
}
Figure