Multiclass fMRI data decoding and visualization using supervised self-organizing maps

Lars Hausfeld, Giancarlo Valente, Elia FormisanoNeuroImage 2014 · DOI · OpenReview

TL;DR

This paper introduces a novel approach using supervised self-organizing maps (SSOMs) for decoding and visualizing complex, multi-class fMRI data. SSOMs offer a powerful way to understand brain activity patterns beyond simple binary classifications, providing insights into the underlying data topology.

What Problem Does This Paper Address?

Traditional fMRI analysis often relies on univariate methods that examine each voxel independently. While Multi-Voxel Pattern Analysis (MVPA) has advanced this by looking at patterns across voxels, most MVPA techniques struggle with datasets involving more than two experimental conditions. These "multiclass" problems are typically broken down into numerous binary comparisons, which can be computationally intensive and obscure the overall picture of brain representation. Furthermore, existing decoding methods often only report classification accuracy, neglecting the valuable information contained in the spatial organization of brain activity. This paper aims to bridge this gap by proposing a method that can directly handle multiclass fMRI decoding and, crucially, visualize the resulting data topology.

Methods

The core of this paper is the application of Supervised Self-Organizing Maps (SSOMs) for fMRI data analysis. SSOMs are a type of neural network that maps high-dimensional data onto a low-dimensional grid (typically 2D) while preserving the topological relationships of the original data. The "supervised" aspect means that class labels are incorporated directly into the training process, allowing SSOMs to perform multiclass classification inherently.

Key methodological contributions include:

  • Direct Multiclass Decoding: Unlike many existing methods that require converting multiclass problems into multiple binary ones, SSOMs handle more than two classes directly.
  • Data Topology Visualization: SSOMs provide a visual representation of how different brain activity patterns (voxels) cluster and relate to each other, offering insights into the underlying representational structure. This is achieved by projecting the trained SSOM onto principal components.
  • Multi-Split and Multi-Subject SSOMs: To generalize findings across different cross-validation splits or individual subjects, the authors developed methods for creating "general" SSOMs. This involves aligning individual SSOMs into a common space and then matching their nodes.
  • k-Best Matching Units (k-BMUs): For classification, instead of relying on a single best-matching unit, the authors use the top 10 BMUs to accumulate evidence, leading to more robust predictions.

Datasets

The authors evaluated their SSOM approach using:

  • Simulated fMRI Data:
    • A 3-class dataset was simulated with varying signal-to-noise (SNR) and contrast-to-noise (CNR) ratios to test the method's performance under different noise conditions.
  • An 8-class simulation was also performed to mimic the complexity of the visual object categories dataset.
  • Real fMRI Data:
    • Speaker Identification Study: Newly collected fMRI data from 5 participants listening to vocalizations from three different speakers. This was a challenging, 3-class decoding task.
  • Visual Object Categories Study (Haxby et al., 2001): A publicly available dataset where participants viewed images of eight different object categories. This dataset is a common benchmark for MVPA studies.

Key Insights and Ideas

  • Beyond Accuracy: The Power of Visualization: The paper strongly emphasizes that simply reporting classification accuracy misses crucial information about how the brain represents information. SSOM visualizations reveal the structure of these representations, showing which categories are easily distinguishable and which are confusable.
  • Multiclass is Not Just More Binary: Directly handling multiclass problems with SSOMs simplifies the analysis pipeline and potentially offers a more holistic view of brain activity compared to numerous pairwise comparisons.
  • Generalizing Across Splits and Subjects: The development of multi-split and multi-subject SSOMs is a significant step towards understanding common neural representations that transcend individual experimental runs or participants.
  • SSOMs as a Complementary Tool: While not always outperforming SVMs in raw accuracy, SSOMs provide unique insights into data topology that other methods lack, making them valuable for exploratory data analysis.

Key Results

  • Simulations: SSOMs performed comparably to SVMs for small to medium feature sets (100-600 voxels) and better than kNN classifiers, especially in challenging conditions. For very large feature sets (>800 voxels), SVMs generally outperformed SSOMs.
  • Speaker Identification: SSOMs achieved similar classification performance to SVMs and kNN. Notably, the SSOM analysis correctly predicted that the female speaker was easier to decode than the two male speakers, aligning with behavioral observations.
  • Visual Object Categories: SSOMs achieved high classification accuracy, outperforming kNN and showing similar performance to linear SVMs. The SSOM visualization revealed clear distinctions between categories like faces and houses, and suggested an animate/inanimate or natural/man-made ordering principle for object representations.

Limitations and Future Work

  • Parameter Optimization: The authors note that SSOM parameters (T and U) were set based on simulations rather than optimized via nested cross-validation. Automating this selection process would be a valuable future direction.
  • Comparison to Other Algorithms: While SSOMs were compared to kNN and SVMs, the authors suggest extending the comparison to other inherently multivariate algorithms like Gaussian Processes, sparse logistic regression, and neural networks.
  • Computational Cost: While SSOMs offer advantages, the multi-split and multi-subject SSOM creation can be computationally intensive.
  • Feature Selection: The paper used ensemble feature selection. Further exploration of how different feature selection strategies interact with SSOMs could be beneficial.
  • Visualization Interpretation: While SSOMs provide visualizations, interpreting these complex maps, especially for highly confusable classes, still requires careful consideration.

Summary updated

Abstract

Highlights•Supervised self-organizing maps (SSOMs) for fMRI multiclass decoding•SSOMs provide intuitive visual accounts of pattern topology.•SSOMs are well-suited for region-of interest or searchlight decoding analysis.
BibTeX
@article{hausfeld2014multiclass,
  author = {Lars Hausfeld and Giancarlo Valente and Elia Formisano},
  title = {{Multiclass fMRI data decoding and visualization using supervised self-organizing maps}},
  journal = {NeuroImage},
  year = {2014},
  doi = {10.1016/j.neuroimage.2014.02.006},
}
Figure