A Multichannel 2D Convolutional Neural Network Model for Task-Evoked fMRI Data Classification

Jinlong Hu, Yuezhen Kuang, Bin Liao, Cao Li-jie, Shoubin Dong, Ping LiComputational 2019 · DOI

TL;DR

This paper introduces M2D CNN, a novel multichannel 2D Convolutional Neural Network designed to classify task-evoked functional Magnetic Resonance Imaging (fMRI) data. By processing 3D fMRI data through multiple 2D CNNs operating on different brain planes and then integrating their learned features, M2D CNN achieves superior classification accuracy while being more computationally efficient and less prone to overfitting than existing 3D CNN approaches.

What Problem Does This Paper Address?

Functional Magnetic Resonance Imaging (fMRI) is a powerful tool for understanding brain activity during specific tasks. Task-evoked fMRI data captures the brain's response to stimuli, generating complex 3D time-series data. The challenge lies in accurately classifying these responses to understand the intricate relationships between brain activity, behavior, and cognitive tasks. Traditional machine learning methods like SVM struggle with the high dimensionality of fMRI data. While Convolutional Neural Networks (CNNs) have shown promise, existing 2D CNNs can lose crucial spatial information from 3D data, and 3D CNNs, though capturing full spatial context, are computationally expensive and prone to overfitting due to their large number of parameters. This paper aims to bridge this gap by developing a CNN architecture that balances high accuracy with computational efficiency and robustness against overfitting for task-evoked fMRI classification.

Methods

The core innovation is the M2D CNN model. Instead of directly processing the 3D fMRI data, M2D CNN first transforms each 3D brain image into three distinct sets of multichannel 2D images. Each set represents the brain data sliced along one of the three anatomical planes (coronal, sagittal, axial). These three sets of 2D images are then fed into three separate 2D CNN architectures running in parallel. Each 2D CNN independently extracts features from its respective plane. The outputs from these three parallel CNNs are then flattened and concatenated, creating a rich feature vector that captures information from all three planes. This combined feature vector is then fed into a fully connected neural network for final classification. This approach leverages the strengths of 2D CNNs (efficiency, feature extraction) while compensating for the loss of spatial information by integrating features from multiple orthogonal views.

Datasets

The study utilizes data from the Human Connectome Project (HCP), specifically the motor task fMRI dataset. This dataset consists of 995 subjects, each providing 4D fMRI data (91x109x91 voxels over 284 time-series frames). The data was preprocessed to extract mean percent signal change (Mean PSC) for each voxel relative to a baseline derived from fixation blocks, resulting in 3D brain images representing task-evoked activity. For experiments, datasets were formed using 200, 500, and all 995 subjects, resulting in 2000, 5000, and 9950 samples, respectively. Five-fold cross-validation was employed for evaluation.

Key Insights and Ideas

  • Bridging 2D and 3D CNNs: The M2D CNN offers a clever compromise between the computational efficiency of 2D CNNs and the spatial completeness of 3D CNNs. By treating different planes as separate "channels" for 2D CNNs, it effectively reconstructs a 3D understanding without the full computational burden of 3D convolutions.
  • Multichannel 2D Input: The novel idea of transforming a 3D volume into multiple 2D "multichannel" images, each representing a different anatomical plane, is key. This allows existing, well-understood 2D CNN architectures to be applied effectively to 3D data.
  • Overfitting Mitigation: The M2D CNN demonstrates a significant advantage in alleviating data overfitting compared to 3D CNNs. This is attributed to its smaller number of parameters, making it more robust when dealing with limited fMRI datasets.
  • CNNs Outperform SVM: The study reinforces the general finding that CNNs, due to their ability to learn hierarchical features, significantly outperform traditional methods like SVM on high-dimensional fMRI data.

Key Results

  • Highest Accuracy: The M2D CNN achieved the highest classification accuracy of 83.20% ± 2.29% on the full dataset of 995 subjects, outperforming all other tested models, including 3D CNN (82.34% ± 1.27%) and 3D SepConv (80.44% ± 1.16%).
  • Computational Efficiency: While 3D CNN models offer good accuracy, they are computationally costly. M2D CNN, with its parallel 2D CNN structure, achieves comparable or better performance with a significantly lower parameter count (6,355,717 for M2D CNN vs. 45,174,181 for 3D CNN).
  • Reduced Overfitting: Figure 8 clearly illustrates that M2D CNN exhibits steadier decreases in both training and validation losses, indicating a reduced tendency to overfit compared to 3D CNN, which shows more fluctuating validation losses.
  • Robustness Across Sample Sizes: M2D CNN consistently performed well across different dataset sizes (200, 500, and 995 subjects), demonstrating its scalability and reliability.

Limitations and Future Work

The authors acknowledge two primary limitations:

  • Task Specificity: The study focused exclusively on a motor task. Future work should explore the M2D CNN's performance on fMRI data from cognitive tasks (e.g., memory, language, vision).
  • Dataset Scale: While 9,950 samples from 995 subjects is substantial, it is still considered relatively small compared to the computational demands of deep learning models.

Future research directions include applying the M2D CNN to larger resting-state and task-evoked fMRI datasets across various cognitive domains to further validate its efficacy and explore its potential in a broader range of neuroimaging applications.

Summary updated

Abstract

Deep learning models have been successfully applied to the analysis of various functional MRI data. Convolutional neural networks (CNN), a class of deep neural networks, have been found to excel at extracting local meaningful features based on their shared-weights architecture and space invariance characteristics. In this study, we propose M2D CNN, a novel multichannel 2D CNN model, to classify 3D fMRI data. The model uses sliced 2D fMRI data as input and integrates multichannel information learned from 2D CNN networks. We experimentally compared the proposed M2D CNN against several widely used models including SVM, 1D CNN, 2D CNN, 3D CNN, and 3D separable CNN with respect to their performance in classifying task-based fMRI data. We tested M2D CNN against six models as benchmarks to classify a large number of time-series whole-brain imaging data based on a motor task in the Human Connectome Project (HCP). The results of our experiments demonstrate the following: (i) convolution operations in the CNN models are advantageous for high-dimensional whole-brain imaging data classification, as all CNN models outperform SVM; (ii) 3D CNN models achieve higher accuracy than 2D CNN and 1D CNN model, but 3D CNN models are computationally costly as any extra dimension is added in the input; (iii) the M2D CNN model proposed in this study achieves the highest accuracy and alleviates data overfitting given its smaller number of parameters as compared with 3D CNN.
BibTeX
@inproceedings{hu2019multichannel,
  author = {Jinlong Hu and Yuezhen Kuang and Bin Liao and Cao Li-jie and Shoubin Dong and Ping Li},
  title = {{A Multichannel 2D Convolutional Neural Network Model for Task-Evoked fMRI Data Classification}},
  booktitle = {Computational},
  year = {2019},
  doi = {10.1155/2019/5065214},
}
Figure