PAIR Comparison between Two Within-Group Conditions of Resting-State fMRI Improves Classification Accuracy

Zhen Zhou, Jianbao Wang, Yu‐Feng Zang, Gang PanFrontiers in Neuroscience 2018 · DOI

TL;DR

This paper introduces a novel "PAIR" method for analyzing resting-state fMRI (RS-fMRI) data, treating paired conditions (like eyes-open vs. eyes-closed) as a single unit rather than independent states. This approach significantly improves classification accuracy, especially for within-subject study designs, by reducing inter-subject variability and better capturing condition-specific brain activity patterns.

What Problem Does This Paper Address?

Functional magnetic resonance imaging (fMRI), particularly resting-state fMRI (RS-fMRI), is a powerful tool for understanding brain activity. Machine learning techniques are increasingly used to classify individuals into groups (e.g., patients vs. controls) or to differentiate between different brain states. However, a major hurdle in RS-fMRI analysis is the "curse of dimensionality" – a vast number of brain voxels (features) compared to a relatively small number of subjects. This often leads to poor generalization of classification models to new data.

Furthermore, most existing classification studies treat different experimental conditions as independent groups. This is problematic for within-subject designs, where the same individuals experience multiple conditions (e.g., eyes-open vs. eyes-closed resting states). Analyzing these conditions independently fails to account for the inherent correlation and shared variance within an individual, potentially leading to suboptimal classification performance. This paper addresses the challenge of improving classification accuracy in RS-fMRI, specifically for within-subject designs, by proposing a new method that leverages the paired nature of experimental conditions.

Methods

The core innovation of this paper is the PAIR method for feature extraction. Instead of treating "eyes closed" (EC) and "eyes open" (EO) as two separate classes, the PAIR method treats them as a paired comparison within each subject.

Here's a breakdown of the key steps:

  1. Feature Metric: The study uses Amplitude of Low-Frequency Fluctuation (ALFF) as the primary metric to quantify spontaneous brain activity. ALFF measures the signal fluctuations within a specific frequency band (0.01–0.08 Hz, and other sub-bands).
  2. Dimensionality Reduction (Paired t-tests): For each dataset, paired t-tests are performed within subjects to identify brain regions showing significant differences between EC and EO conditions. This step helps reduce the high dimensionality of the data by focusing on relevant brain areas.
  3. PAIR Feature Extraction: The crucial step involves creating difference maps. Half of the subjects contribute an "EC minus EO" (EC-EO) ALFF map, while the other half contribute an "EO minus EC" (EO-EC) ALFF map. These difference maps, representing the contrast between the two conditions for each subject, are then used as features. This approach effectively captures the difference in brain activity between the two states for each individual.
  4. Classification: A Support Vector Machine (SVM) is employed as the classifier. The SVM is trained on these PAIR-extracted features to distinguish between the two conditions.
  5. UNPAIR Method (for comparison): The authors also implement a "UNPAIR" method, which treats EC and EO as independent classes, using standard two-sample t-tests for feature selection. This serves as a baseline to demonstrate the superiority of the PAIR method.
  6. Feature Selection Strategies: Three feature selection strategies were explored:
  7. Peak-ALFF: ALFF value at the peak voxel of significant regions.
  8. Mean-ALFF: Mean ALFF within a spherical region of interest (ROI) centered at the peak voxel.
  9. All-ALFF: ALFF values of all voxels within the spherical ROI.

Datasets

The study utilized two independent RS-fMRI datasets:

  • Dataset-1: 31 healthy adult participants (aged 19-31 years) from Hangzhou Normal University.
  • Dataset-2: 46 healthy adult participants (aged 19-31 years) from a publicly available dataset (Beijing: Eyes Open Eyes Closed Study, downloaded from http://fcon_1000.projects.nitrc.org/indi/IndiPro.html).

Both datasets included RS-fMRI scans acquired under two conditions: eyes closed (EC) and eyes open (EO). The study performed both within-dataset validation (using leave-one-out cross-validation, LOOCV) and cross-dataset validation to assess the generalizability of their findings.

Key Insights and Ideas

  • Leveraging Within-Subject Variance: The PAIR method's success stems from its ability to reduce inter-subject variability. By focusing on the difference between conditions within each subject, it effectively isolates the signal related to the condition change, making it more robust than treating conditions independently.
  • Paired Design is Crucial: The stark difference in classification accuracy between the PAIR and UNPAIR methods highlights the importance of accounting for the paired nature of experimental conditions in within-subject designs. Simply treating them as independent groups misses valuable information.
  • Frequency Band Matters: While the conventional 0.01-0.08 Hz band yielded good results, higher frequency bands (0.073–0.198 Hz and 0.198–0.25 Hz) showed even better prediction accuracy in some cases. This suggests that different frequency ranges might capture distinct aspects of brain activity relevant to condition changes.
  • Eyeballs and Blinks: The inclusion of an "eyeball" region in the analysis was motivated by the expectation that eye blinks (more prevalent in EO) would influence ALFF. Indeed, the eyeball region showed significant differences and contributed to classification, particularly in higher frequency bands.
  • T-values and SVM Weights Correlate: The study found a strong correlation between the t-values from the initial paired t-tests (used for dimensionality reduction) and the weights assigned by the SVM classifier. This provides empirical support for using t-tests as a valid feature selection method in this context, although the authors caution against "double dipping."

Key Results

  • High Classification Accuracy with PAIR: The PAIR method achieved high mean classification accuracy: 91.40% for dataset-1 and 92.75% for dataset-2 in the 0.01–0.08 Hz band.
  • Excellent Cross-Dataset Generalization: Cross-dataset validation was also impressive, with mean accuracies of 94.93% (dataset-1 to dataset-2) and 90.32% (dataset-2 to dataset-1) in the 0.01–0.08 Hz band.
  • Significant Improvement over UNPAIR: The UNPAIR method (treating EC and EO as independent) performed substantially worse, with mean accuracies of 69.89% (dataset-1) and 82.97% (dataset-2) within datasets, and much lower cross-dataset accuracies (around 64%).
  • Feature Selection Strategies: The three feature selection methods (Peak-ALFF, Mean-ALFF, All-ALFF) showed similar performance, with Mean-ALFF being slightly more stable.
  • Brain Regions of Interest: The sensorimotor cortex (SMC) and visual cortex consistently showed high contributions to classification accuracy across datasets and validation schemes.

Limitations and Future Work

  • Random Grouping: The study randomly divided subjects into EC-EO and EO-EC groups for feature extraction. While they hypothesized this wouldn't affect results, they acknowledge the lack of theoretical proof and suggest future work to explore mathematical solutions for this grouping.
  • Limited Dimensionality Reduction Methods: Only t-tests were used for dimensionality reduction. The authors suggest exploring other methods to potentially improve accuracy and consistency.
  • Low Frequency Band Consistency: The 0.01–0.027 Hz frequency band showed lower consistency in cross-dataset validation, which the authors find difficult to interpret and suggest further investigation.
  • Complex T-value/Weight Relationship: While a correlation was observed, the relationship between t-values and SVM weights was complex, particularly after removing certain brain regions. Further in-depth investigation is recommended.
  • Generalizability to Other Within-Group Designs: The PAIR method's success in differentiating EC and EO suggests its potential applicability to other within-group neuroimaging studies (e.g., comparing different task conditions in task-fMRI or longitudinal studies), which warrants future exploration.

Summary updated

Abstract

Classification approaches have been increasingly applied to differentiate patients and normal controls using resting-state functional magnetic resonance imaging data (RS-fMRI). Although most previous classification studies have reported promising accuracy within individual datasets, achieving high levels of accuracy with multiple datasets remains challenging for two main reasons: high dimensionality, and high variability across subjects. We used two independent RS-fMRI datasets (<i>n</i> = 31, 46, respectively) both with eyes closed (EC) and eyes open (EO) conditions. For each dataset, we first reduced the number of features to a small number of brain regions with paired <i>t</i>-tests, using the amplitude of low frequency fluctuation (ALFF) as a metric. Second, we employed a new method for feature extraction, named the PAIR method, examining EC and EO as paired conditions rather than independent conditions. Specifically, for each dataset, we obtained EC minus EO (EC-EO) maps of ALFF from half of subjects (<i>n</i> = 15 for dataset-1, <i>n</i> = 23 for dataset-2) and obtained EO-EC maps from the other half (<i>n</i> = 16 for dataset-1, <i>n</i> = 23 for dataset-2). A support vector machine (SVM) method was used for classification of EC RS-fMRI mapping and EO mapping. The mean classification accuracy of the PAIR method was 91.40% for dataset-1, and 92.75% for dataset-2 in the conventional frequency band of 0.01-0.08 Hz. For cross-dataset validation, we applied the classifier from dataset-1 directly to dataset-2, and vice versa. The mean accuracy of cross-dataset validation was 94.93% for dataset-1 to dataset-2 and 90.32% for dataset-2 to dataset-1 in the 0.01-0.08 Hz range. For the UNPAIR method, classification accuracy was substantially lower (mean 69.89% for dataset-1 and 82.97% for dataset-2), and was much lower for cross-dataset validation (64.69% for dataset-1 to dataset-2 and 64.98% for dataset-2 to dataset-1) in the 0.01-0.08 Hz range. In conclusion, for within-group design studies (e.g., paired conditions or follow-up studies), we recommend the PAIR method for feature extraction. In addition, dimensionality reduction with strong prior knowledge of specific brain regions should also be considered for feature selection in neuroimaging studies.
BibTeX
@inproceedings{zhou2018pair,
  author = {Zhen Zhou and Jianbao Wang and Yu‐Feng Zang and Gang Pan},
  title = {{PAIR Comparison between Two Within-Group Conditions of Resting-State fMRI Improves Classification Accuracy}},
  booktitle = {Frontiers in Neuroscience},
  year = {2018},
  doi = {10.3389/fnins.2017.00740},
}
Figure