A Novel 3D Approach with a CNN and Swin Transformer for Decoding EEG-Based Motor Imagery Classification

Xin Deng, Huaxiang Huo, Lijiao Ai, Daijiang Xu, Chenhui LiSensors 2025 · DOI

TL;DR

This paper introduces "ConSwinFormer," a novel 3D deep learning architecture that combines Convolutional Neural Networks (CNNs) and Swin Transformers to improve the accuracy of decoding motor imagery (MI) from electroencephalography (EEG) signals. By transforming EEG data into a 3D representation and leveraging the strengths of both CNNs for local feature extraction and Swin Transformers for global context, the model achieves state-of-the-art performance on a challenging BCI dataset.

What Problem Does This Paper Address?

Brain-computer interfaces (BCIs) hold immense promise for individuals with motor impairments, allowing them to control external devices through brain signals. Motor imagery (MI), the act of imagining movement, is a key BCI paradigm. However, decoding EEG signals for MI is difficult due to their high dimensionality and low signal-to-noise ratio (SNR). Traditional deep learning methods often struggle to effectively capture both the intricate spatial relationships between EEG electrodes and the long-term temporal dependencies within the signals. This limitation hinders the accuracy and reliability of MI-BCI systems.

Methods

The core innovation is the ConSwinFormer model, which processes EEG data in a novel 3D format. Here's a breakdown:

  • 3D EEG Representation: Raw EEG signals are first filtered into different frequency bands (delta, theta, alpha, beta, gamma). This multi-channel frequency data is then structured into a 3D tensor, where dimensions represent frequency bands, time points, and a spatial arrangement of electrodes. This allows for simultaneous processing of spatio-temporal and frequency information.
  • CNN for Local Features: A CNN module is employed to extract local spatio-temporal features. It uses 1D convolutions for temporal patterns and 2D convolutions for spatial relationships across electrodes within specific time windows.
  • 3D Swin Transformer for Global Features: Following the CNN, a 3D Swin Transformer module takes over. This architecture is designed to capture long-range dependencies and global context by employing shifted window-based self-attention mechanisms. This allows the model to understand how different parts of the brain activity relate to each other across time and space.
  • Data Augmentation: To combat the common issue of small datasets in BCI research and prevent overfitting, the authors employ "Segment and Reassemble" (S&R) techniques and random noise addition. This dynamically augments the training data.
  • Relative Position Bias: A 3D relative position bias is introduced into the self-attention mechanism to further enhance the model's ability to capture spatial relationships.

Datasets

The primary dataset used for evaluation is the BCI Competition IV-2a (BCICIV-2a) dataset. This is a widely recognized and challenging benchmark in MI classification research. It comprises EEG data from 9 subjects, each performing four distinct motor imagery tasks (left hand, right hand, both feet, tongue). The dataset is notable for including two sessions per subject, collected at different times, to assess cross-session generalization.

Key Insights and Ideas

  • 3D is Key: Transforming EEG into a 3D representation is crucial for effectively capturing the complex spatio-temporal-frequency characteristics of brain signals.
  • CNN + Swin Synergy: Combining the local feature extraction power of CNNs with the global context understanding of Swin Transformers creates a potent hybrid model for EEG analysis.
  • Windowed Attention for Efficiency: The Swin Transformer's shifted window mechanism allows for efficient computation of global dependencies without the quadratic complexity of standard self-attention.
  • Data Augmentation Matters: Dynamic data augmentation is vital for improving model generalization and reducing overfitting, especially with limited BCI datasets.
  • Interpretability Through Visualization: Techniques like t-SNE and Grad-CAM help visualize how the model processes data and which brain regions it focuses on, offering insights into the underlying neural processes.

Key Results

  • State-of-the-Art Accuracy: The ConSwinFormer model achieved an average classification accuracy of 83.99% on the BCICIV-2a dataset.
  • Superior Performance: This accuracy is significantly higher than several existing advanced methods, including traditional machine learning approaches (e.g., FBCSP, PSCSP) and other deep learning models (e.g., Multi-branch 3D CNN, Transformer-based methods, CMO-CNN).
  • Individual Subject Performance: The model demonstrated strong performance across individual subjects, with some reaching accuracies as high as 98.60% (Subject S03) and 96.51% (Subject S07).
  • Robustness: Ablation studies showed that the combined CNN and Swin Transformer approach significantly outperformed models using only CNN or only Swin Transformer, highlighting the complementary nature of the components.

Limitations and Future Work

  • Subject Variability: The authors acknowledge that accuracy varies significantly across subjects, suggesting that individual differences in EEG signals pose a challenge. Future work aims to investigate personalized adjustment strategies.
  • Data Augmentation: While effective, the authors plan to explore more advanced data augmentation techniques to further mitigate issues with small datasets and reduce overfitting.
  • Cross-Session Performance: The dataset's cross-session nature highlights the need for models that generalize well over time.
  • Real-World Noise: The current experiments were conducted on clean datasets. Future work will test the model under more realistic noisy conditions.
  • Broader Applications: The authors suggest exploring the model's applicability to other BCI tasks and its potential for multitask learning and transfer learning. They also envision integrating the model into real-time BCI systems for practical applications in communication and rehabilitation.

Summary updated

Abstract

Motor imagery (MI) is a crucial research field within the brain-computer interface (BCI) domain. It enables patients with muscle or neural damage to control external devices and achieve movement functions by simply imagining bodily motions. Despite the significant clinical and application value of MI-BCI technology, accurately decoding high-dimensional and low signal-to-noise ratio (SNR) electroencephalography (EEG) signals remains challenging. Moreover, traditional deep learning approaches exhibit limitations in processing EEG signals, particularly in capturing the intrinsic correlations between electrode channels and long-distance temporal dependencies. To address these challenges, this research introduces a novel end-to-end decoding network that integrates convolutional neural networks (CNNs) and a Swin Transformer, aiming at enhancing the classification accuracy of the MI paradigm in EEG signals. This approach transforms EEG signals into a three-dimensional data structure, utilizing one-dimensional convolutions along the temporal dimension and two-dimensional convolutions across the EEG electrode distribution for initial spatio-temporal feature extraction, followed by deep feature exploration using a 3D Swin Transformer module. Experimental results show that on the BCI Competition IV-2a dataset, the proposed method achieves 83.99% classification accuracy, which is significantly better than the existing deep learning methods. This finding underscores the efficacy of combining a CNN and Swin Transformer in a 3D data space for processing high-dimensional, low-SNR EEG signals, offering a new perspective for the future development of MI-BCI. Future research could further explore the applicability of this method across various BCI tasks and its potential clinical implementations.
BibTeX
@inproceedings{deng2025novel,
  author = {Xin Deng and Huaxiang Huo and Lijiao Ai and Daijiang Xu and Chenhui Li},
  title = {{A Novel 3D Approach with a CNN and Swin Transformer for Decoding EEG-Based Motor Imagery Classification}},
  booktitle = {Sensors},
  year = {2025},
  doi = {10.3390/s25092922},
}
Figure