Single-Trial Detection and Classification of Event-Related Optical Signals for a Brain–Computer Interface Application

Nicole Chiou, Mehmet Günal, Oluwasanmi Koyejo, David Perpetuini, Antonio Maria Chiarelli, Kathy A. Low, Monica Fabiani, Gabriele GrattonBioengineering 2024 · DOI

TL;DR

This paper pioneers the use of deep learning, specifically Convolutional Neural Networks (CNNs), for single-trial classification of Event-Related Optical Signals (EROS) – a high-resolution brain imaging technique. They achieved promising results, with subject-specific models trained on EROS phase data reaching an average accuracy of 62.8%, demonstrating the potential of EROS for brain-computer interfaces (BCIs).

What Problem Does This Paper Address?

Brain-Computer Interfaces (BCIs) aim to enable direct communication between the brain and external devices. While Electroencephalography (EEG) is a common BCI modality, its low spatial resolution limits performance. Event-Related Optical Signals (EROS) offer a compelling alternative, boasting high spatial and temporal resolution, allowing for detailed brain activity monitoring. However, a significant hurdle for EROS in BCI applications has been the challenge of reliably classifying individual brain signals (single trials) due to their inherent low signal-to-noise ratio (SNR). Previous EROS-based BCI research has largely relied on averaging data across many trials to improve SNR, which slows down communication speed. This paper tackles the problem of achieving accurate single-trial classification of EROS, specifically for motor responses, by leveraging the power of deep learning.

Methods

The core of this study is the application of a modified Convolutional Neural Network (CNN) architecture, adapted from successful EEG-based BCI models. This CNN is designed to automatically learn spatiotemporal features directly from raw EROS data, eliminating the need for manual feature engineering. Key aspects of their approach include:

  • Depthwise Separable Convolutions: This architectural choice, inspired by computer vision techniques, allows for efficient learning of spatial and temporal features while reducing the number of parameters, making it suitable for limited data scenarios.
  • Subject-Specific Training: Recognizing inter-subject variability, the researchers developed and evaluated three distinct training paradigms:
  • Montage-Specific Classification: Training a separate model for each recording montage.
  • Cross-Montage Classification: Training a single model on data from all montages combined.
  • Montage-Specific Classification with Pre-training: Pre-training a model on data from all montages except the target one, then fine-tuning it on the target montage.
  • Input Signal Exploration: The study investigated the effectiveness of using only phase delay data, only intensity data, or a combination of both as input to the CNN.
  • Feature Relevance Analysis: Using DeepLIFT, they visualized which parts of the EROS signal (specific time intervals and spatial locations) were most influential in the CNN's predictions, offering insights into the underlying brain activity.

Datasets

The study utilized data from a previously published experiment [8] involving 12 healthy, right-handed participants. These participants performed a two-choice reaction time task requiring left or right-hand responses. EROS data were collected using a high-density recording montage covering the motor cortex. For each participant, data were acquired across four different, but overlapping, recording montages. This resulted in a dataset of single-trial EROS recordings, with a maximum of 240 trials available per subject per montage for training.

Key Insights and Ideas

  • Phase Data Dominance: EROS phase data proved significantly more discriminative for motor response classification than intensity data, aligning with prior research suggesting intensity is more susceptible to noise.
  • Deep Learning's Advantage: CNNs excel at automatically learning complex spatiotemporal features from EROS, bypassing the need for manual feature extraction, which is a significant advantage for this high-resolution but noisy signal.
  • Montage-Specific Superiority: For single-trial classification, training a model specifically on data from a single montage (the "best montage" identified by validation accuracy) yielded the highest performance. This suggests that while multi-montage approaches are useful, aligning signals across different montages remains a challenge for EROS.
  • Data Quality Matters: The number of viable voxels and the average number of channels per voxel within a montage positively correlated with classification accuracy, highlighting the importance of good sensor placement and coverage over discriminative brain regions.
  • Right Hemisphere Focus: The right hemisphere showed a stronger correlation between signal quality and classification accuracy, potentially due to its more specialized role in contralateral movements for right-handed individuals.
  • Broadband is Best: Using a broad frequency band (0.1-12 Hz) for phase data was more effective than narrow bands, suggesting that the CNN can effectively learn relevant temporal features across frequencies without explicit domain expertise.

Key Results

  • Average Accuracy: The best-performing setup, using a montage-specific CNN trained on 0.1-12 Hz filtered phase data, achieved an average held-out test accuracy of 62.8% (95% CI: [0.574, 0.682]) and an AUROC of 0.674 (95% CI: [0.615, 0.733]) across 12 participants.
  • Above Chance Performance: A significant majority of subjects achieved classification accuracy above the chance level (50%).
  • Intensity vs. Phase: Subject-specific classifiers trained on EROS phase data outperformed those trained on intensity data.
  • Dual-Input Not Superior: Combining phase and intensity data did not significantly improve classification accuracy compared to using phase data alone.
  • Feature Relevance: Analysis revealed that discriminative signals originate from the motor cortex, particularly in the right hemisphere, with key contributions occurring around 179 ms and 410 ms after the response.

Limitations and Future Work

  • Accuracy Threshold: While promising, the achieved accuracy of 62.8% is below the 70% threshold often considered essential for effective BCI communication. This is attributed, in part, to the limited number of trials available per subject-montage combination.
  • Data Augmentation: The authors suggest that data augmentation strategies could be employed to simulate larger datasets, which would likely improve model performance.
  • Subject-Specific Alignment: Future work could explore using subject-specific structural MRI data to more accurately align optical channels across different montages, potentially improving multi-montage approaches.
  • Cross-Subject Classification: Investigating the development of cross-subject classifiers would be a valuable next step.
  • Dual-Modality Further Investigation: While not superior in this study, the potential of dual-modality (phase + intensity) analysis warrants further investigation with larger datasets.
  • Pre-selection of Montages: For multi-montage training paradigms, pre-selecting high-performing montages could further enhance results.

Summary updated

Abstract

Event-related optical signals (EROS) measure fast modulations in the brain's optical properties related to neuronal activity. EROS offer a high spatial and temporal resolution and can be used for brain-computer interface (BCI) applications. However, the ability to classify single-trial EROS remains unexplored. This study evaluates the performance of neural network methods for single-trial classification of motor response-related EROS. EROS activity was obtained from a high-density recording montage covering the motor cortex during a two-choice reaction time task involving responses with the left or right hand. This study utilized a convolutional neural network (CNN) approach to extract spatiotemporal features from EROS data and perform classification of left and right motor responses. Subject-specific classifiers trained on EROS phase data outperformed those trained on intensity data, reaching an average single-trial classification accuracy of around 63%. Removing low-frequency noise from intensity data is critical for achieving discriminative classification results with this measure. Our results indicate that deep learning with high-spatial-resolution signals, such as EROS, can be successfully applied to single-trial classifications.
BibTeX
@inproceedings{chiou2024single,
  author = {Nicole Chiou and Mehmet Günal and Oluwasanmi Koyejo and David Perpetuini and Antonio Maria Chiarelli and Kathy A. Low and Monica Fabiani and Gabriele Gratton},
  title = {{Single-Trial Detection and Classification of Event-Related Optical Signals for a Brain–Computer Interface Application}},
  booktitle = {Bioengineering},
  year = {2024},
  doi = {10.3390/bioengineering11080781},
}
Figure