Cross-Subject Motor Imagery Decoding by Transfer Learning of Tactile ERD

Yucun Zhong, Lin Yao, Gang Pan, Yueming WangIEEE Transactions on Neural Systems and Rehabilitation Engineering 2024 · DOI

TL;DR

This paper introduces a novel method for calibrating Brain-Computer Interfaces (BCIs) that use motor imagery (MI). Instead of relying on users to mentally imagine movements, the researchers used passive tactile stimulation to elicit brain patterns. This "tactile calibration" method achieved comparable decoding performance to traditional MI calibration but required significantly fewer trials, making it more practical, especially for clinical applications.

What Problem Does This Paper Address?

Brain-Computer Interfaces (BCIs) that rely on motor imagery (MI) are powerful tools for communication and control, particularly for individuals with severe motor impairments. However, a major hurdle is the calibration process. MI tasks are abstract and spontaneous, leading to noisy EEG signals and requiring a substantial amount of data to train accurate decoding models. This lengthy calibration is inconvenient, especially for patients. Furthermore, ensuring users are performing MI correctly is challenging, and some individuals may struggle to generate clear brain patterns. Existing methods to reduce calibration effort, like using passive movements or electrical stimulation, have their own practical limitations. This paper aims to address the need for a more efficient, reliable, and user-friendly calibration method for cross-subject MI decoding.

Methods

The core innovation is using tactile stimulation to generate brain patterns for BCI calibration. The researchers designed an experimental paradigm with two main blocks:

  • Motor Imagery (MI) Block: Subjects mentally imagined left or right hand movements.
  • Tactile Stimulation (TS) Block: Subjects passively received tactile stimulation on their left or right wrist, focusing on sensing the sensation.

Crucially, the EEG data collected during the TS block was used to train or fine-tune models that could then decode actual MI data from a different subject (cross-subject decoding).

They employed two main algorithmic approaches for evaluation:

  1. Common Spatial Pattern (CSP) with Linear Discriminant Analysis (LDA): A classic BCI feature extraction and classification pipeline. They specifically explored how tactile data performed compared to traditional MI data for training CSP-LDA models.
  2. Deep Transfer Learning (EEGsym model): A state-of-the-art deep learning model pre-trained on large public EEG datasets. This model was then fine-tuned using either MI data or the novel tactile stimulation data.

The study also incorporated Euclidean Alignment to help align data distributions across subjects and tasks, aiming to improve transfer learning effectiveness.

Datasets

The study used data collected from 12 healthy participants (6 male, 6 female).

  • In-house collected data: EEG data was recorded from these 12 participants during both the MI and TS tasks. This data was used for both training and testing.
  • Public EEG datasets: Data from five publicly available datasets (Physionet, OpenBMI, Kaya2018, Meng2019, Stieger2021) were used for pre-training the deep transfer learning model (EEGsym).

Key Insights and Ideas

  • Tactile Stimulation as a Proxy for MI: The most significant insight is that passive tactile stimulation can elicit brain patterns (specifically Event-Related Desynchronization/Synchronization - ERD/ERS) that are remarkably similar to those induced by motor imagery. This similarity is observed in both the spatial distribution of brain activity (concentrated in the sensorimotor cortex) and frequency bands (alpha and beta).
  • Reduced Calibration Burden: The tactile approach offers a more explicit and controllable way to elicit brain patterns compared to the abstract nature of MI. This translates to a need for fewer trials to achieve a satisfactory decoding performance, significantly reducing the calibration workload for users.
  • Enhanced Discriminative Information: Analysis of R² values (indicating discriminative information) showed that tactile stimulation data provided even more discriminative information than MI data across various frequency bands, potentially explaining the superior performance with fewer trials.
  • Feasibility for Diverse Populations: The tactile method's independence from actual movement makes it a promising candidate for populations where traditional MI calibration is difficult or impossible, such as individuals with severe motor impairments or the elderly.

Key Results

  • Comparable Performance: Both the CSP-LDA and deep transfer learning methods achieved decoding accuracies with tactile calibration that were comparable to traditional MI calibration. For CSP-LDA, accuracies were around 74.48% for MI calibration and 76.35% for tactile calibration (no significant difference). For the deep transfer learning model (EEGsym), accuracies were 75.10% for MI and 73.96% for tactile.
  • Fewer Trials Needed: Tactile calibration demonstrated a significant advantage when the number of trials was limited. For instance, achieving 70% accuracy required only about 5 trials per class (110 total) with tactile calibration, compared to at least 10 trials per class (220 total) for traditional MI calibration.
  • Faster Fine-tuning: In the deep transfer learning context, models fine-tuned with tactile data converged significantly faster than those fine-tuned with MI data, suggesting the tactile data is more distinct and easier for the model to learn from.
  • Cross-Subject Correlation: A strong correlation (r=0.97) was found between tactile calibration performance and traditional MI calibration performance across subjects, indicating that the patterns learned from tactile stimulation generalize well.

Limitations and Future Work

  • Online Performance: The study focused on offline cross-subject calibration. The authors acknowledge that evaluating the performance of this tactile calibration method in real-time online BCI applications is a crucial next step.
  • Elderly Population: While promising for diverse populations, the authors suggest further investigation into the effectiveness of this method for the elderly, who may have reduced tactile sensitivity.
  • Database Development: Building a comprehensive database of EEG data induced by tactile stimulation is proposed as a future direction to further enhance transfer learning capabilities.
  • Real-time Decoding: The ultimate goal is to use this tactile data to fine-tune pre-trained models for real-time decoding, enabling a truly hands-off calibration process.

Summary updated

Abstract

For Brain-Computer Interface (BCI) based on motor imagery (MI), the MI task is abstract and spontaneous, presenting challenges in measurement and control and resulting in a lower signal-to-noise ratio. The quality of the collected MI data significantly impacts the cross-subject calibration results. To address this challenge, we introduce a novel cross-subject calibration method based on passive tactile afferent stimulation, in which data induced by tactile stimulation is utilized to calibrate transfer learning models for cross-subject decoding. During the experiments, tactile stimulation was applied to either the left or right hand, with subjects only required to sense tactile stimulation. Data from these tactile tasks were used to train or fine-tune models and subsequently applied to decode pure MI data. We evaluated BCI performance using both the classical Common Spatial Pattern (CSP) combined with the Linear Discriminant Analysis (LDA) algorithm and a state-of-the-art deep transfer learning model. The results demonstrate that the proposed calibration method achieved decoding performance at an equivalent level to traditional MI calibration, with the added benefit of outperforming traditional MI calibration with fewer trials. The simplicity and effectiveness of the proposed cross-subject tactile calibration method make it valuable for practical applications of BCI, especially in clinical settings.
BibTeX
@article{zhong2024cross,
  author = {Yucun Zhong and Lin Yao and Gang Pan and Yueming Wang},
  title = {{Cross-Subject Motor Imagery Decoding by Transfer Learning of Tactile ERD}},
  journal = {IEEE Transactions on Neural Systems and Rehabilitation Engineering},
  year = {2024},
  doi = {10.1109/tnsre.2024.3358491},
}
Figure