CoSupFormer : A Contrastive Supervised learning approach for EEG signal Classification

Davy Darankoum, Chloé Habermacher, Julien Volle, Sergei GrudininarXiv 2025 · DOI · arXiv

TL;DR

CoSupFormer is a novel deep-learning framework that achieves state-of-the-art EEG signal classification by combining multi-scale convolutional encoding with a "gated" global attention mechanism. By leveraging a hybrid loss function that joins supervised and contrastive learning, the model effectively filters out noisy channels and generalizes across different species (humans and mice) and neurological conditions.

What Problem Does This Paper Address?

Electroencephalography (EEG) is a gold mine for understanding brain states, but it is notoriously "messy" data. Raw EEG signals are plagued by low signal-to-noise ratios, artifacts (like eye blinks or muscle movements), and significant variability between different subjects or even different recording sessions.

Traditional approaches often rely on hand-crafted features or standard Transformers that treat all recording channels as equally important. However, in real-world clinical or pharmacological settings, some channels might be corrupted or entirely non-informative. The challenge lies in building a model that can autonomously identify relevant features across multiple frequency scales while ignoring the "noise" inherent in multi-channel recordings.

Methods

The authors introduce CoSupFormer, which stands out through three architectural innovations:

  1. Multi-scale Dual-Path Encoder: Instead of a single convolutional layer, the model uses two parallel branches. One uses small kernels to catch high-frequency, fine-grained patterns; the other uses large, dilated convolutions to capture low-frequency oscillations over wider time windows without exploding the parameter count.
  2. Gated Global Attention: Most Transformers for time-series look at interactions within a single channel or across channels at a fixed time. CoSupFormer’s attention module models interactions across both channels and time steps simultaneously. Crucially, it adds a gating mechanism (a learned sigmoid mask) that can completely suppress the signal from noisy or irrelevant "patches" of data before they reach the final decision layers.
  3. CoSup Loss: The model is trained using a hybrid objective. It combines standard Cross-Entropy (for classification) with NT-Xent (a contrastive loss). This forces the model to not only predict the right label but also to ensure that embeddings of the same brain state are clustered together in the latent space, while different states are pushed apart.

Datasets

The researchers evaluated CoSupFormer on five distinct setups:

  • TDBrain: A large human dataset focused on Parkinson’s disease vs. healthy controls.
  • ADFTD: A public human dataset for Alzheimer’s Disease and Frontotemporal Dementia.
  • MACO: A private, massive (1,536 hours) dataset of mouse EEG used to classify the effects of different pharmacological compounds (antidepressants vs. antipsychotics).
  • Noisy Variants: The authors created "TDBrain+Noise" and "ADFTD+Noise" by injecting synthetic Gaussian noise into a subset of channels to specifically test the gating mechanism's robustness.

Key Insights and Ideas

  • The Power of "Hard" Gating: While standard attention "downweights" unimportant information, the authors found that an explicit gating mechanism is superior for EEG. It acts as an automated quality-control layer, effectively "unplugging" noisy electrodes on the fly.
  • Contrastive Learning as a Regularizer: The hybrid loss function proved vital for generalization. By using contrastive learning, the model learns a more structured latent space (as seen in their PCA visualizations), which helps it remain accurate even when the input signal is degraded.
  • Translational Versatility: One of the most impressive "aha" moments is the model's performance on the MACO (mouse) dataset. Despite the biological differences between species, the same architecture successfully extracted meaningful patterns from rodent brain activity, suggesting the model captures fundamental principles of electrophysiology.

Key Results

CoSupFormer consistently outperformed established baselines like iTransformer, PatchTST, and MedFormer:

  • Clean Data: On the TDBrain dataset, it achieved 95.81% accuracy, significantly higher than the next best baseline (MedFormer at 89.62%).
  • Robustness to Noise: On the "TDBrain+Noise" dataset, CoSupFormer maintained a 94.98% accuracy, while other models saw their performance crater (MedFormer dropped to 49.79%).
  • Pharmacological Classification: On the challenging MACO mouse dataset, CoSupFormer reached 74.93% accuracy, nearly doubling the performance of several transformer-based baselines which struggled to exceed 38%.

Limitations and Future Work

While CoSupFormer is highly effective for EEG, the authors note a few areas for growth:

  • Domain Specificity: The model was designed and tested exclusively on EEG. Its performance on other medical time-series, such as ECG (heart) or EMG (muscle) signals, remains untested.
  • Data Requirements: The authors suggest that in extremely low-data regimes, the complex hybrid loss might require very careful hyperparameter tuning to avoid performance drops.
  • Interpretability: While the gating mechanism tells us which channels are ignored, further work is needed to map the learned "multi-scale" features back to specific biological rhythms (like Alpha or Theta waves) to satisfy clinical interpretability needs.

Summary updated

Abstract

Electroencephalography signals (EEGs) contain rich multi-scale information crucial for understanding brain states, with potential applications in diagnosing and advancing the drug development landscape. However, extracting meaningful features from raw EEG signals while handling noise and channel variability remains a major challenge. This work proposes a novel end-to-end deep-learning framework that addresses these issues through several key innovations. First, we designed an encoder capable of explicitly capturing multi-scale frequency oscillations covering a wide range of features for different EEG-related tasks. Secondly, to model complex dependencies and handle the high temporal resolution of EEGs, we introduced an attention-based encoder that simultaneously learns interactions across EEG channels and within localized {\em patches} of individual channels. We integrated a dedicated gating network on top of the attention encoder to dynamically filter out noisy and non-informative channels, enhancing the reliability of EEG data. The entire encoding process is guided by a novel loss function, which leverages supervised and contrastive learning, significantly improving model generalization. We validated our approach in multiple applications, ranging from the classification of effects across multiple Central Nervous System (CNS) disorders treatments to the diagnosis of Parkinson's and Alzheimer's disease. Our results demonstrate that the proposed learning paradigm can extract biologically meaningful patterns from raw EEG signals across different species, autonomously select high-quality channels, and achieve robust generalization through innovative architectural and loss design.
BibTeX
@misc{darankoum2025cosupformer,
  author = {Davy Darankoum and Chloé Habermacher and Julien Volle and Sergei Grudinin},
  title = {{CoSupFormer : A Contrastive Supervised learning approach for EEG signal Classification}},
  year = {2025},
  eprint = {2509.20489},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2509.20489},
  howpublished = {arXiv},
}
Figure