Motor Imagery Classification Using Hybrid Cross-Subject Class Mixup and Selective Attention

Jiwon Kim, Chae-Won Lee, Jong-Seok LeearXiv 2026 · arXiv

TL;DR

This paper introduces a novel hybrid framework for Brain-Computer Interfaces (BCI) that significantly improves how models generalize across different users. By combining Cross-Subject Class Mixup (CSCM)—which independently mixes EEG and fNIRS signals from different subjects—with a Selective Attention mechanism that respects the different timescales of brain activity, the authors achieved state-of-the-art accuracy in motor imagery classification.

What Problem Does This Paper Address?

The "holy grail" of Brain-Computer Interfaces is a system that works for a new user right out of the box. However, brain signals are notoriously "noisy" and vary significantly from person to person—a challenge known as inter-subject variability.

While researchers have turned to hybrid systems that combine EEG (which captures fast electrical activity) and fNIRS (which measures slower blood-oxygen changes), two major hurdles remain:

  1. Generalization: Models often overfit to the specific quirks of the training subjects and fail on new ones.
  2. Information Fusion: EEG and fNIRS operate on vastly different physiological timescales. Finding an architecture that effectively fuses these "apples and oranges" without introducing redundant noise is difficult.

Methods

The authors propose a two-pronged solution:

1. Cross-Subject Class Mixup (CSCM)

Standard "Mixup" is a data augmentation technique that creates new training samples by interpolating between existing ones. CSCM takes this further for the multimodal setting. For a given "anchor" trial (e.g., imagining a left-hand movement), the system independently selects an EEG donor from Subject B and an fNIRS donor from Subject C, both performing the same task.

Figure 1: EEG–fNIRS training-input construction: (a) no mix- ing, (b) within-subject mixing, and (c) CSCM. Mixing uses same-class donors at a 50:50 ratio and preserves the anchor label. CSCM selects EEG and fNIRS donors independently. Letters (A, B, C) denote subjects, and signal subscripts denote trial (i, j, k) and subject identities, respectively.

By mixing these independently, the model is forced to learn robust, class-relevant features rather than relying on subject-specific signal patterns.

2. Selective Attention

The model uses a Transformer-based architecture but applies attention "selectively." * EEG: Receives causal temporal self-attention to capture rapid changes over time. * fNIRS: Bypasses internal temporal attention updates. Because fNIRS signals are much slower, the authors found that complex temporal self-attention was unnecessary and potentially harmful. * Fusion: A classification (CLS) token acts as a "bridge," reading information from both the EEG and fNIRS branches to make the final prediction.

Figure 2: Overview of the proposed framework. CSCM is applied only during training, while inference uses the original paired signals. The modality-specific encoders produce 19 EEG and 25 fNIRS tokens, forming a 45-token sequence with CLS. Learned temporal embeddings are added before layer normalization, dropout, and four Transformer blocks. The final CLS representation is used for left/right MI classification.

Datasets

The study utilized a widely recognized public hybrid EEG–fNIRS dataset (Shin et al., 2017).

  • Subjects: 28 healthy participants.
  • Task: Binary motor imagery (imagining left-hand vs. right-hand movement).
  • Evaluation: A rigorous Leave-One-Subject-Out (LOSO) protocol, ensuring the model is tested on a completely "unseen" person's brain activity.

Key Insights and Ideas

  • Independent Modality Mixing is Superior: One of the most surprising findings was that picking EEG and fNIRS donors from different subjects worked better than picking them from the same donor. This suggests that decoupling the modalities during training prevents the model from memorizing specific cross-modal artifacts.
  • Less is More for fNIRS: In the "Selective Attention" design, disabling fNIRS-to-fNIRS temporal attention actually improved performance. This highlights that the slow hemodynamic response of fNIRS doesn't benefit from the fine-grained temporal modeling that EEG requires.
  • The CLS Token is an Effective Integrator: By disabling direct EEG-to-fNIRS temporal connections and forcing all communication through the CLS token, the model achieved a cleaner fusion of the two data streams.
Figure 3: Representative temporal-attention configura- tions with fixed CLS-based integration: (a) Full/Full, (b) Causal/Causal, and (c) Causal/OFF (Selective Attention). Rows represent queries and columns represent keys and val- ues. The 19 EEG and 25 fNIRS tokens are shown schematically. CLS attends to both modalities, while EEG tokens can attend to CLS. Selective Attention bypasses fNIRS attention updates while retaining residual and feed-forward processing. Direct cross-modal temporal-token attention is disabled through- out.

Key Results

The proposed framework outperformed five established baselines in the field:

  • Balanced Accuracy (BA): 76.85%, exceeding the strongest baseline (EFMLNet) by 3.81 percentage points.
  • Macro-F1 Score: 76.58%, an improvement of 4.05 percentage points over the best baseline.
  • Ablation Success: Statistical tests confirmed that both CSCM and Selective Attention contributed significantly to the performance boost, with their combination yielding the best results.

Limitations and Future Work

While the results are promising, the authors note a few areas for expansion:

  • Dataset Diversity: The current evaluation is limited to a single public dataset. Testing on more diverse populations and different BCI paradigms (like mental arithmetic or multi-class movement) is necessary to prove broader applicability.
  • Real-time Implementation: The study focuses on offline classification. Moving toward real-time, "closed-loop" BCI applications would be the next logical step for clinical or assistive technology use.
  • Task Complexity: The current work handles binary classification (left vs. right). Future research could explore finer-grained tasks, such as individual finger movements or wrist flexion.

Summary updated

Abstract

Motor imagery (MI) enables a brain-computer interface (BCI) to decode movement intentions without overt movement. Hybrid electroencephalography (EEG) and functional near-infrared spectroscopy (fNIRS) classification combines complementary signals, but inter-subject variability and the design of information exchange between modalities affect generalization to new subjects. We propose a framework that combines Cross-Subject Class Mixup (CSCM) with Selective Attention. CSCM aims to construct new training data to make the model robust against inter-subject variability. It independently selects EEG and fNIRS donor trials from other training subjects within the anchor class and mixes each modality while preserving the original label. Selective Attention provides an effective way to simultaneously model the temporal signal characteristics and fuse the two modalities. It retains causal temporal self-attention for EEG and uses a classification token (CLS) to integrate both modalities, without fNIRS attention updates or direct cross-modal temporal-token attention. Leave-one-subject-out (LOSO) evaluation across 28 subjects on a public EEG-fNIRS dataset yields a mean balanced accuracy (BA) of 76.85% and a macro-averaged F1 score (Macro-F1) of 76.58%. The proposed model achieves the highest mean performance among five reimplemented baselines. We also examine the contributions of CSCM and Selective Attention through modality-specific mixing and donor-policy comparisons, temporal-policy comparisons, interventions on direct cross-modal paths, and component ablations.
BibTeX
@misc{kim2026motor,
  author = {Jiwon Kim and Chae-Won Lee and Jong-Seok Lee},
  title = {{Motor Imagery Classification Using Hybrid Cross-Subject Class Mixup and Selective Attention}},
  year = {2026},
  eprint = {2611.20853},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.20853},
}
Figure