Pattern classification precedes region-average hemodynamic response in early visual cortex

Peter J. Kohler, Sergey V. Fogelson, Eric A. Reavis, Ming Meng, J. Swaroop Guntupalli, Michael Hanke, Yaroslav O. Halchenko, Andrew C. Connolly, James V. Haxby, Peter U. TseNeuroImage 2013 · DOI · OpenReview

TL;DR

This study reveals that we can decode visual information from brain activity in early visual cortex (V1) using fMRI before the average blood flow response in that region even changes from baseline. Furthermore, the peak of this decodable information occurs earlier than the peak of the blood flow response, suggesting that fMRI can capture neural signals much faster than previously assumed.

What Problem Does This Paper Address?

Functional Magnetic Resonance Imaging (fMRI) measures brain activity indirectly by detecting changes in blood flow, known as the hemodynamic response (HR). This HR is relatively slow, typically starting seconds after neural activity and peaking several seconds later. This temporal lag makes it challenging to pinpoint exactly when information about a stimulus is truly available in the brain based on fMRI data. Traditional analyses (univariate) look at average responses, but more advanced techniques like Multi-Voxel Pattern Analysis (MVPA) can detect subtle patterns across many voxels. While MVPA can often detect information that univariate methods miss, its temporal dynamics relative to the slow HR were not fully understood. Specifically, could MVPA detect information before the average HR even deviates from baseline, and could the peak of this information precede the peak HR? This paper investigates this temporal dissociation between neural information (as decoded by MVPA) and the hemodynamic response in early visual cortex.

Methods

The researchers employed a slow event-related fMRI design with rapid temporal sampling (TR of 739 ms) to capture the hemodynamic response with high temporal resolution. Participants viewed images of faces and houses while performing a one-back task.

Key methodological components include:

  • Multi-Voxel Pattern Analysis (MVPA): This was the core analysis technique. They used two classifiers:
  • Linear Support Vector Machines (SVM)

    • Sparse Multinomial Logistic Regression (SMLR) These classifiers were trained to distinguish between face and house stimuli based on patterns of voxel activity within specific regions of interest (ROIs).
  • Leave-one-run-out cross-validation: This robust approach ensured that the classifier was tested on data it hadn't seen during training, providing a more reliable estimate of classification accuracy.

  • Nonparametric Monte Carlo simulations: Used to determine statistical significance for classification accuracy, establishing a chance level.
  • Region-of-Interest (ROI) definition: ROIs were defined using standard retinotopic mapping techniques and functional localizers for visual areas including V1, V2, V3, V3A/B, V4v, LOC, OFA, FFA, and PPA.
  • Unparameterized and Parameterized Analyses:
    • Unparameterized: Directly compared the timecourses of classification accuracy and region-average HR at each time point.
  • Parameterized: Fitted double gamma functions to the HR and classification timecourses to extract parameters like onset, peak, and duration for more precise comparisons.
  • Feature Selection and Voxel Sensitivity Analysis: These methods were used to identify which voxels contributed most to early classification, helping to understand the neural basis of the observed effects.
  • Smoothing Analysis: Investigated the effect of spatial smoothing on classification accuracy to understand the scale of the information being decoded.

Datasets

The study used fMRI data collected from 11 healthy participants. The visual stimuli consisted of images of human faces and houses, drawn from a stimulus set previously used by Haxby et al. (2001). No new benchmark datasets were introduced.

Key Insights and Ideas

  • Early Information in V1: The most striking finding is that information about visual stimuli can be decoded from the patterns of fMRI activity in V1 before the average hemodynamic response in V1 deviates from baseline. This suggests that the hemodynamic response is not the sole determinant of when information is detectable via fMRI.
  • Temporal Dissociation: The peak of classification accuracy in V1 occurs earlier than the peak of the region-average hemodynamic response. This challenges the assumption that peak information availability always aligns with peak blood flow.
  • V1 Specificity: These "early onset" and "early peak" effects are largely confined to early visual cortex (V1 and to some extent V2). Higher-level visual areas do not show this phenomenon, and in some cases, exhibit the opposite temporal relationship.
  • Not Due to ROI Size: The early classification in V1 is not simply a result of V1 being a larger region than others. A combined ROI of comparable size to V1 (comprising LOC, OFA, FFA, PPA) did not show early classification.
  • Small Number of Informative Voxels: In V1, classification is driven by a small subset of voxels, suggesting that specific voxels can carry information earlier than the bulk of the region.
  • Information Decay: In V1, the duration of above-chance classification is shorter than the duration of the region-average HR. This implies that the early-decoded information fades relatively quickly, even as the blood flow response persists.

Key Results

  • Early Onset: In V1, classification accuracy became significantly above chance approximately 1 second before the region-average HR deviated from baseline. This effect was also observed in V2 with one classifier.
  • Early Peak: In V1, the peak classification accuracy occurred approximately 0.5-1 second before the peak of the region-average HR. This effect was also seen in some other areas, but less consistently.
  • Duration: Classification duration in V1 was shorter than the HR duration, while in PPA, it was longer.
  • Smoothing Effects: While smoothing generally reduced the magnitude of the effects, both early onset and early peak classification persisted even with significant spatial smoothing, suggesting the information is not solely dependent on very fine-grained, sub-voxel details.
  • Voxel Contribution: Feature selection identified that a small number of voxels (around 2% of the V1 mask) were sufficient to drive classification in V1.

Limitations and Future Work

  • Stimulus Specificity: The authors acknowledge that the observed effects might be stimulus-specific. Future work should explore whether these findings generalize to other types of visual stimuli and other brain areas.
  • Mechanism of Early Classification: The exact neural mechanisms driving this early classification remain unclear. The authors speculate on possibilities like initial dips in the HR or balanced positive/negative responses, but definitive answers are lacking.
  • Generalizability of SMLR vs. SVM: The discrepancy in how well SMLR weights correlated with F-scores compared to SVM suggests potential differences in how these classifiers utilize information, which warrants further investigation.
  • Future Directions: The authors suggest that these findings have methodological implications for fMRI study design and analysis, encouraging the development of techniques that can leverage information from the earliest components of the hemodynamic response. They also propose exploring similar dissociations in other brain regions and with different cognitive tasks.

Summary updated

BibTeX
@article{kohler2013pattern,
  author = {Peter J. Kohler and Sergey V. Fogelson and Eric A. Reavis and Ming Meng and J. Swaroop Guntupalli and Michael Hanke and Yaroslav O. Halchenko and Andrew C. Connolly and James V. Haxby and Peter U. Tse},
  title = {{Pattern classification precedes region-average hemodynamic response in early visual cortex}},
  journal = {NeuroImage},
  year = {2013},
  doi = {10.1016/j.neuroimage.2013.04.019},
}
Figure