Validating Deep Neural Networks for Online Decoding of Motor Imagery Movements from EEG Signals

Qu Xingwei, Zied Tayeb, Juri Fedjaev, Nejla Ghaboosi, Christoph Richter, Lukas Everding, Xingwei Qu, Yingyu Wu, Gordon Cheng, Jörg ConradtSensors 2019 · DOI

TL;DR

This paper introduces and validates three deep learning models (LSTM, pCNN, RCNN) for decoding motor imagery (MI) movements from EEG signals, aiming to improve the accuracy and robustness of brain-computer interfaces (BCIs). The authors demonstrate that their proposed "pragmatic" CNN (pCNN) achieves high accuracy with significantly lower computational cost than other deep learning models, enabling real-time control of a robotic arm.

What Problem Does This Paper Address?

Brain-computer interfaces (BCIs) offer a lifeline for individuals with motor disabilities, enabling them to communicate and control devices using their thoughts. Motor imagery (MI) BCIs, which translate imagined movements into control signals from EEG data, have shown promise. However, progress has stagnated due to the inherent challenges of EEG signals: low signal-to-noise ratio (SNR) and high variability across trials and individuals. Traditional machine learning methods for EEG decoding often rely on hand-crafted features, which are difficult to extract and can be suboptimal due to the non-stationary nature of EEG. This paper addresses the need for more robust and accurate EEG decoding methods by exploring the potential of deep learning models that can learn features directly from raw EEG data, bypassing the need for manual feature engineering.

Methods

The researchers developed and evaluated three deep learning architectures for decoding motor imagery movements:

  • Long Short-Term Memory (LSTM): Chosen for its ability to learn long-term dependencies in time-series data, an LSTM network with one hidden layer was implemented.
  • Pragmatic Convolutional Neural Network (pCNN): This novel CNN architecture is designed to be computationally efficient while maintaining high accuracy. It consists of three convolutional blocks, each with a convolutional layer, batch normalization, and max-pooling, followed by a fully connected layer with a softmax activation.
  • Recurrent Convolutional Neural Network (RCNN): This model combines RNN and CNN principles, using recurrent convolutional layers (RCLs) to capture both spatial and temporal information.

In addition to their proposed models, the authors also reimplemented and benchmarked two existing deep learning models: a shallow CNN (sCNN) and a deep CNN (dCNN) from previous work. For comparison, they also evaluated five traditional machine learning classifiers (KNN, DT, LR, NB, QLDA) using hand-crafted features like logarithmic band power (log-BP), common spatial patterns (CSP), and wavelet coefficients.

Datasets

The study utilized two datasets:

  • Own Recorded EEG Data: Data was collected from 20 healthy, right-handed subjects performing left and right hand motor imagery tasks. This dataset was recorded using 32 active electrodes (reduced to C3, C4, and Cz) at 250 Hz.
  • BCI Competition IV Dataset 2b: This publicly available dataset was used to further validate the models' performance on independent data. It consists of EEG data from nine subjects performing left and right hand motor imagery.

Key Insights and Ideas

  • Deep Learning Outperforms Traditional Methods: The study consistently shows that deep learning models, particularly the CNN-based approaches, achieve significantly higher classification accuracies compared to traditional machine learning methods, even when the latter use hand-crafted features.
  • The "Pragmatic" Approach: The pCNN model offers a compelling balance between accuracy and computational efficiency. Its ability to achieve high performance with fewer parameters makes it a more suitable candidate for real-time BCI applications where computational resources are often limited.
  • End-to-End Learning is Key: The success of deep learning models in directly processing raw EEG (or spectrograms) highlights the power of end-to-end learning, eliminating the need for laborious and potentially suboptimal manual feature engineering.
  • Spectrograms as Input: Converting EEG time-series data into spectrograms provides a rich, time-frequency representation that CNNs can effectively leverage to learn discriminative features.
  • Real-Time Control Achieved: The successful real-time control of a robotic arm using the pCNN-based BCI demonstrates the practical viability of these deep learning approaches for assistive technologies.

Key Results

  • Own Dataset:
    • The dCNN and pCNN models achieved the highest mean accuracies, outperforming other developed classifiers.
  • The pCNN model achieved a mean accuracy of 84.24% (±14.69%) across all subjects.
  • The dCNN model achieved a mean accuracy of 92.28% (±1.69%), but with significantly higher computational demands.
  • The LSTM model achieved a mean accuracy of 66.2% (±7.21%), performing similarly to the sCNN.
  • The RCNN model achieved a mean accuracy of 77.72% (±6.50%).
  • BCI Competition IV Dataset 2b:
    • The pCNN model demonstrated strong performance, achieving a mean accuracy of 91.63% on this dataset.
  • The dCNN model also performed well with a mean accuracy of 95.72%.
  • Robotic Arm Control: The pCNN model enabled real-time control of a robotic arm, with a noticeable delay of approximately 1.4 seconds attributed to the CNN processing.

Limitations and Future Work

  • Overfitting: The authors acknowledge that overfitting can be a concern, particularly with smaller datasets. They employed techniques like early stopping and dropout to mitigate this.
  • Data Augmentation: While data augmentation (cropping) was used to increase the training set size, the authors note that the overall amount of data used for training was still limited.
  • Computational Cost: While the pCNN is more efficient than dCNN, deep learning models in general can still be computationally intensive. The authors suggest that future work could explore transfer learning to pre-train models on larger datasets.
  • RCNN Optimization: The RCNN model showed potential but could be further improved by optimizing its architecture, possibly by combining CNN and LSTM elements as seen in other research.
  • NPU Integration: The authors envision future BCI devices being powered by neural processing units (NPUs) rather than CPUs, enabling faster, more power-efficient, and compact systems.

Summary updated

BibTeX
@article{xingwei2019validating,
  author = {Qu Xingwei and Zied Tayeb and Juri Fedjaev and Nejla Ghaboosi and Christoph Richter and Lukas Everding and Xingwei Qu and Yingyu Wu and Gordon Cheng and Jörg Conradt},
  title = {{Validating Deep Neural Networks for Online Decoding of Motor Imagery Movements from EEG Signals}},
  journal = {Sensors},
  year = {2019},
  doi = {10.3390/s19010210},
}
Figure