Delayed knowledge transfer: Cross-modal knowledge transfer from delayed stimulus to EEG for continuous attention detection based on spike-represented EEG signals

Pengfei Sun, Jorg De Winne, Malu Zhang, Paul Devos, Dick BotteldoorenNeural Networks 2025 · DOI · OpenReview

Abstract

Decoding cognitive states from Electroencephalography (EEG) signals is complicated by inherent biological processing delays and the high energy demands of traditional neural networks. This paper introduces the Delayed Knowledge Transfer (DKT) framework, which utilizes biologically inspired Spiking Neural Networks (SNNs) to improve attention detection. By employing a Teacher-Student learning paradigm with a novel, learnable Delay Layer, the framework aligns external audiovisual stimuli with delayed neural responses. Experimental results on the "WithMeAttention" dataset demonstrate a 3% accuracy improvement over baseline models and a 36.39% reduction in energy consumption, highlighting the potential for efficient, real-time Brain-Computer Interfaces (BCIs).

Introduction

Brain-Computer Interfaces (BCIs) often rely on EEG to monitor attention, but mapping these signals to external events is hindered by two primary factors. First, the biological lag, known as Event-Related Potentials (ERPs), creates a temporal misalignment between a stimulus and the brain's electrical response. Second, traditional Artificial Neural Networks (ANNs) are computationally expensive, limiting their utility in wearable, battery-powered BCI devices.

Event-Related Potentials (ERPs) illustrating the temporal delay in neural responses to auditory and visual stimuli.

Methods

The DKT framework addresses these challenges through a Teacher-Student (T-S) paradigm built on Spiking Neural Networks (SNNs). SNNs utilize Leaky Integrate-and-Fire (LIF) neurons that only compute when a voltage threshold is reached, mimicking the energy efficiency of the human brain.

The DKT Framework

The system consists of two primary components:

  1. Teacher Model: Trained on visual and auditory sequences to extract features of "attention-grabbing" stimuli.
  2. Student Model: An SNN that processes raw 64-channel EEG signals.
Overview of the Delayed Knowledge Transfer (DKT) framework, showing the Teacher model, Delay Layer, and EEG Student model.

The core innovation is the learnable Delay Layer. This layer mathematically determines the optimal time offset (τ) to align stimulus features from the Teacher with the corresponding brain responses in the Student. To handle the non-differentiable nature of discrete time delays, the authors employed a Straight-Through Estimator (STE) during backpropagation.

Dataset: WithMeAttention

The researchers developed the "WithMeAttention" benchmark, involving 20 participants performing a digit-memorization task under four conditions: purely visual (C1), rhythmic visual (C2), visual with non-rhythmic audio (C3), and rhythmic visual with rhythmic audio (C4).

Results and Discussion

The DKT framework achieved a peak accuracy of 59.73% using a 0.17-second decision window, outperforming established models like EEGNet and CNNNet.

Key Findings

  • Rhythmic Synchronization: Rhythmic stimuli (C2 and C4) significantly enhanced decoding accuracy. Rhythm appears to act as a "neural hook," synchronizing attentional oscillations and producing more predictable EEG patterns.
  • Bimodal Processing Delays: The learned delays typically clustered around 200ms and 450ms, reflecting the biological reality that auditory information is generally processed faster than complex visual data.
  • Energy Efficiency: The spiking implementation reduced computational costs by 36.39% (measured in picojoules) compared to standard ANNs, making it highly suitable for edge BCI hardware.

Conclusion

The Delayed Knowledge Transfer framework successfully bridges the gap between external stimuli and internal neural states by accounting for biological latency. By combining cross-modal knowledge transfer with energy-efficient spiking neurons, DKT provides a robust foundation for the next generation of real-time, wearable attention-monitoring systems. Future work will focus on refining individual variability in audiovisual binding and integrating more complex attention mechanisms within the SNN architecture.

Reference: Sun, P., et al. (2024). Delayed Knowledge Transfer: Cross-Modal Knowledge Transfer from Delayed Stimulus to EEG for Continuous Attention Detection Based on Spike-Represented EEG Signals. Preprint submitted to Elsevier.

Summary updated

Abstract

Decoding visual and auditory stimuli from brain activities, such as electroencephalography (EEG), offers promising advancements for enhancing machine-to-human interaction. However, effectively representing EEG signals remains a significant challenge. In this paper, we introduce a novel Delayed Knowledge Transfer (DKT) framework that employs spiking neurons for attention detection, using our experimental EEG dataset. This framework extracts patterns from audiovisual stimuli to model brain responses in EEG signals, while accounting for inherent response delays. By aligning audiovisual features with EEG signals through a shared embedding space, our approach improves the performance of brain–computer interface (BCI) systems. We also present WithMeAttention, a multimodal dataset designed to facilitate research in continuously distinguishing between target and distractor responses. Our methodology demonstrates a 3% improvement in accuracy on the WithMeAttention dataset compared to a baseline model that decodes EEG signals from scratch. This significant performance increase highlights the effectiveness of our approach Comprehensive analysis across four distinct conditions shows that rhythmic enhancement of visual information can optimize multi-sensory information processing. Notably, the two conditions featuring rhythmic target presentation – with and without accompanying beeps – achieved significantly superior performance compared to other scenarios. Furthermore, the delay distribution observed under different conditions indicates that our delay layer effectively emulates the neural processing delays in response to stimuli.
BibTeX
@article{sun2025delayed,
  author = {Pengfei Sun and Jorg De Winne and Malu Zhang and Paul Devos and Dick Botteldooren},
  title = {{Delayed knowledge transfer: Cross-modal knowledge transfer from delayed stimulus to EEG for continuous attention detection based on spike-represented EEG signals}},
  journal = {Neural Networks},
  year = {2025},
  doi = {10.1016/j.neunet.2024.107003},
}
Figure