EEG Classification of Motor Imagery Using a Novel Deep Learning Framework

Mengxi Dai, Dezhi Zheng, Rui Na, Shuai Wang, Shuailei Zhang2019 · DOI

TL;DR

This paper introduces a novel deep learning framework, CNN-VAE, for classifying motor imagery (MI) from EEG signals. By combining a Convolutional Neural Network (CNN) for feature extraction with a Variational Autoencoder (VAE) for classification, the framework achieves state-of-the-art performance on benchmark datasets and a newly collected dataset, demonstrating improved accuracy and robustness in brain-computer interfaces.

What Problem Does This Paper Address?

Brain-computer interfaces (BCIs) that rely on motor imagery (MI) – the mental simulation of movement – hold immense potential for communication and control, especially for individuals with severe motor impairments. However, the practical application of these BCIs is often limited by the accuracy and robustness of EEG signal classification. EEG signals are inherently noisy and complex, making it challenging to reliably decode a user's intended movement. Existing methods, while improving, still face challenges in handling the intricate spatio-temporal patterns within EEG data and achieving consistent performance across different users and conditions. This paper aims to address these limitations by proposing a more effective deep learning approach for MI classification.

Methods

The core of this paper is the proposed CNN-VAE framework. This approach integrates two powerful deep learning architectures:

  • Convolutional Neural Network (CNN): The CNN is used for initial feature extraction. Instead of treating EEG signals as raw time series, the authors transform them into 2D images that encode time, frequency, and channel information. This is achieved using the Short-Time Fourier Transform (STFT) to capture spectral content within specific frequency bands (mu and beta) and then combining these with spatial information from different electrodes. The CNN then learns spatial, spectral, and temporal patterns from these input images. Notably, the convolutional filters are applied 1D along the time axis, while the height of the kernel matches the input image's height (representing electrode-frequency combinations).
  • Variational Autoencoder (VAE): The features extracted by the CNN are then fed into a VAE for classification. The VAE is designed to learn a latent representation of the data and reconstruct it. In this context, the VAE's decoder is configured to output a Gaussian distribution, which is used to fit the distribution of the EEG signals. This allows the VAE to learn a robust, undercomplete representation of the extracted features, effectively capturing the most salient information for classification. The VAE is trained in an unsupervised manner, followed by supervised fine-tuning.

To compare their proposed method, the authors also implemented and evaluated standalone CNN, CNN-SAE (CNN with Stacked Autoencoders), and VAE models.

Datasets

The study utilized two distinct datasets for training and evaluation:

  1. BCI Competition IV dataset 2b: This is a well-established benchmark dataset for MI classification. It includes EEG data from nine subjects performing left- and right-hand MI tasks, recorded with three channels (C3, Cz, C4) at a 250 Hz sampling rate. The dataset consists of 400 trials per session.
  2. Own Dataset: The authors collected their own dataset from five male subjects (ages 23-30) performing left- and right-hand MI tasks. This dataset was recorded with five channels (C3, C1, Cz, C2, C4) at a 250 Hz sampling rate.

Key Insights and Ideas

  • Transforming EEG into Images: The novel input representation, converting EEG time-series data into 2D images that capture time, frequency, and channel information, is a crucial innovation. This allows CNNs, which excel at image recognition, to effectively process complex EEG patterns.
  • VAE for Robust Classification: Using a VAE with a Gaussian output decoder to fit the EEG signal distribution is a key insight. This approach leverages the known Gaussian-like nature of EEG signals to learn a more robust and interpretable latent representation, leading to improved classification.
  • Synergy of CNN and VAE: The combined CNN-VAE framework demonstrates that the strengths of both architectures can be synergistically leveraged. The CNN extracts rich, multi-dimensional features, and the VAE then uses these features for accurate and robust classification, outperforming individual components.
  • Impact of Channel Number: The study suggests that increasing the number of EEG channels (from three to five) generally leads to improved classification performance, highlighting the importance of spatial information.

Key Results

  • State-of-the-Art Performance: The CNN-VAE framework achieved the best performance on the BCI Competition IV dataset 2b, showing a 3% improvement in average kappa value (0.564) compared to the best existing methods.
  • Strong Performance on Own Dataset: On their newly collected dataset, the CNN-VAE also yielded the best results, with average kappa values of 0.568 for three-electrode data and 0.603 for five-electrode data.
  • Robustness: The CNN-VAE method demonstrated good robustness, with a standard deviation of 0.065 on the BCI Competition IV dataset 2b, indicating consistent performance.
  • Channel Impact: For their own dataset, the five-electrode configuration consistently outperformed the three-electrode setup across all tested methods, with the CNN-VAE achieving the highest kappa value (0.603) with five electrodes.

Limitations and Future Work

  • Low Signal-to-Noise Ratio: The authors acknowledge that EEG signals have a low signal-to-noise ratio, which remains a challenge. Future work will explore new approaches to overcome this.
  • Channel Contribution: While the study shows that more channels improve performance, a deeper dive into the specific contributions of individual channels and their positions is suggested for future research.
  • Filter Contribution: The paper notes that CNN filters contribute significantly to classification performance, but their role in providing information beyond neighboring regions is limited. Further investigation into filter design could be beneficial.
  • Statistical Significance: While the CNN-VAE generally performed best, the authors mention that there were no statistically significant differences among all compared methods in some analyses, suggesting that further refinement or larger datasets might be needed to definitively prove superiority in all scenarios.

Summary updated

BibTeX
@misc{dai2019eeg,
  author = {Mengxi Dai and Dezhi Zheng and Rui Na and Shuai Wang and Shuailei Zhang},
  title = {{EEG Classification of Motor Imagery Using a Novel Deep Learning Framework}},
  year = {2019},
  doi = {10.3390/s19030551},
}
Figure