A novel approach to classify natural grasp actions by estimating muscle activity patterns from EEG signals

Jeong-Hyun Cho, Ji-Hoon Jeong, Dong-Joo Kim, Seong-Whan LeearXiv 2020 · OpenReview

TL;DR

This paper introduces a novel approach to improve brain-computer interface (BCI) accuracy for classifying natural grasp actions by estimating muscle activity patterns from EEG signals. By comparing EEG-derived patterns to simultaneously recorded EMG patterns, the researchers achieved significantly higher classification accuracy for both actual movements and motor imagery, paving the way for more robust BCI applications like robotic control.

What Problem Does This Paper Address?

Brain-computer interfaces (BCIs) hold immense promise for controlling assistive devices, such as upper limb prosthetics, but their practical application is often hindered by low decoding accuracy. This is particularly true for complex motor tasks like hand grasping, which involve intricate neural signals. Existing EEG-based BCI systems struggle to reliably distinguish between various grasp actions, especially during motor imagery (imagining a movement without performing it). This study aims to bridge this gap by proposing a new method to enhance the accuracy of classifying natural grasp actions from EEG signals, with the ultimate goal of enabling more intuitive and precise control of robotic systems.

Methods

The core innovation of this paper lies in its "muscle activity pattern matching" approach. The researchers simultaneously recorded EEG and EMG signals from ten healthy participants performing or imagining three distinct grasp actions (cylindrical, spherical, and lateral).

Here's a breakdown of their method:

  • Data Acquisition: EEG data was collected using 20 electrodes placed on the motor cortex, while EMG data was gathered from seven electrodes targeting specific muscles in the right arm, plus a reference electrode on the elbow.
  • EMG Pattern Estimation: For actual movements, EMG signals were segmented using a sliding window. The Root Mean Square (RMS) value of these segments was calculated. A threshold was established to identify periods of significant muscle activity. This process was repeated for each EMG channel, creating a "muscle activity pattern" represented as a 6x30 image (6 channels x 30 segments).
  • EEG Pattern Estimation: For EEG data, a similar sliding window approach was used. Common Spatial Patterns (CSP) were employed to extract spatial features from the filtered EEG signals. These features were then used to train a Linear Discriminant Analysis (LDA) classifier. Crucially, the labels for training this EEG classifier were derived from the EMG decoding process during actual movements. This allowed them to generate an "estimated pattern" from EEG that aimed to mimic the EMG activity.
  • Classification: The similarity between the estimated EEG patterns and the ground truth EMG patterns was assessed using Mean Squared Error (MSE). The grasp action with the lowest MSE (highest similarity) was classified as the intended action.
  • Motor Imagery: For motor imagery, the EEG classifier trained on actual movement data was re-used, as direct EMG data was not available.

Datasets

The study utilized a custom dataset collected from ten healthy participants (six male, four female, ages 24-33) at Korea University. Participants performed or imagined three specific grasp actions. This dataset was used for both training and evaluating their proposed method, as well as comparing it against existing models. No publicly available datasets were explicitly mentioned as being used.

Key Insights and Ideas

  • EEG as a Proxy for EMG: The most significant insight is the successful demonstration that EEG signals, when processed through a specific pattern-matching framework, can effectively estimate and represent muscle activity patterns. This is a crucial step towards inferring motor intent from non-invasive brain signals.
  • Leveraging Actual Movement Data for Motor Imagery: The ingenious use of EMG-derived patterns from actual movements to train an EEG classifier for motor imagery is a clever workaround for the lack of direct muscle activity data during imagination. While it introduces limitations, it shows promise for improving motor imagery BCI.
  • Importance of Muscle Activity Patterns: The paper strongly suggests that focusing on the patterns of muscle activity, rather than just raw signal amplitudes, is key to improving classification accuracy for complex motor tasks.
  • Subject-Specific Performance: While the average accuracy improved, the results highlight that performance can vary significantly between subjects, indicating the need for personalized BCI calibration.

Key Results

The proposed method demonstrated a substantial improvement in classification accuracy compared to two competitive models (Model I: CSP+LDA, Model II: FBRCSP):

  • Actual Movement:
    • Proposed Method: 63.89% ($\pm$7.54%)
    • Model I: 24.01% higher accuracy than Model I.
  • Model II: 21.59% higher accuracy than Model II.
  • Motor Imagery:
    • Proposed Method: 46.96% ($\pm$15.30%)
    • Model I: 8.60% higher accuracy than Model I.
  • Model II: 5.66% higher accuracy than Model II.

The paper notes that the improvement in motor imagery was less pronounced than for actual movement, but still significant, particularly for specific subjects.

Limitations and Future Work

  • Motor Imagery Accuracy: The authors acknowledge that the classification accuracy for motor imagery is lower than for actual movement. They attribute this to the inability to directly measure EMG during imagination, leading to a less precise EEG-derived pattern.
  • Binary Classifier Limitation: The use of a binary classifier for creating the estimated EEG pattern is identified as a potential bottleneck for motor imagery.
  • Subject Variability: The performance improvement varies across subjects, suggesting that further personalization or more robust decoding models are needed.

Future Work:

  • Improving the Binary Classifier: The authors suggest enhancing the binary classifier used to generate the estimated EEG pattern.
  • Advanced Similarity Comparison: Exploring advanced models, such as deep learning, for the similarity comparison step is proposed to potentially improve accuracy.
  • BCI-Driven Robot Control: The ultimate goal is to apply this method to BCI-driven robotic control for handling everyday objects, which would require further validation and refinement.

Summary updated

Abstract

Developing electroencephalogram (EEG) based brain-computer interface (BCI) systems is challenging. In this study, we analyzed natural grasp actions from EEG. Ten healthy subjects participated in this experiment. They executed and imagined three sustained grasp actions. We proposed a novel approach which estimates muscle activity patterns from EEG signals to improve the overall classification accuracy. For implementation, we have recorded EEG and electromyogram (EMG) simultaneously. Using the similarity of the estimated pattern from EEG signals compare to the activity pattern from EMG signals showed higher classification accuracy than competitive methods. As a result, we obtained the average classification accuracy of 63.89($\pm$7.54)% for actual movement and 46.96($\pm$15.30)% for motor imagery. These are 21.59% and 5.66% higher than the result of the competitive model, respectively. This result is encouraging, and the proposed method could potentially be used in future applications, such as a BCI-driven robot control for handling various daily use objects.
BibTeX
@misc{cho2020novel,
  author = {Jeong-Hyun Cho and Ji-Hoon Jeong and Dong-Joo Kim and Seong-Whan Lee},
  title = {{A novel approach to classify natural grasp actions by estimating muscle activity patterns from EEG signals}},
  year = {2020},
  eprint = {2002.00556},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2002.00556},
}

Loading related content...

Figure