EEG-based video identification using graph signal modeling and graph convolutional neural network

Soobeom Jang, Seong-Eun Moon, Jong-Seok Lee

Abstract

This paper proposes a novel graph signal-based deep learning method for electroencephalography (EEG) and its application to EEG-based video identification. We present new methods to effectively represent EEG data as signals on graphs, and learn them using graph convolutional neural networks. Experimental results for video identification using EEG responses obtained while watching videos show the effectiveness of the proposed approach in comparison to existing methods. Effective schemes for graph signal representation of EEG are also discussed.

Introduction

Electroencephalography (EEG) is a critical tool for monitoring brain activity, yet its analysis remains challenging due to the irregular nature of the data. Unlike the regular grids of pixels in images or the linear sequences of audio, EEG data is captured by sensors placed at specific, non-uniform points on the scalp. The most vital information often resides in the connectivity—the functional and spatial interactions between different brain regions.

While deep learning has transformed fields like computer vision, applying it to EEG is difficult because the number of sensors is relatively low. This research addresses the challenge of modeling sparse brain signals as graphs to perform video identification: the task of matching a subject's neural response to the specific visual stimulus they were watching.

Methodology

The proposed framework transforms raw brainwaves into graph-structured data through a multi-step pipeline:

  1. Feature Extraction: Raw EEG signals are decomposed into eight frequency bands (e.g., $\delta, \theta, \alpha_{low}, \alpha_{high}, \beta_{low}, \beta_{mid}, \beta_{high}, \gamma$). For each band, features such as signal power and entropy are calculated.
  2. Graph Construction: Three connectivity methods were evaluated: correlation-based (functional connectivity), distance-based (physical proximity), and random (baseline).
  3. Sparsification and Inter-band Merging: To reduce noise, only the strongest $k$ connections per electrode are retained. A key innovation is the creation of a "richer" graph by linking electrodes across different frequency bands, allowing the model to learn cross-frequency relationships.
  4. GCNN Architecture: The model utilizes ChebNet, a Graph Convolutional Neural Network that employs Chebyshev polynomials for efficient, localized convolutions on the graph.
Overview of the proposed EEG-based video identification pipeline, from raw signal processing to GCNN classification.

Dataset

The study utilized the DEAP dataset, a benchmark for affective state analysis containing 32-channel EEG data from 32 participants watching 40 one-minute music videos. The 60-second trials were segmented into three-second windows to provide a robust training set.

Results and Discussion

The GCNN approach significantly outperformed traditional machine learning baselines. The highest accuracy achieved was 65.27%, using distance-based graphs with inter-band connectivity and entropy features. In comparison, standard classifiers like Random Forest and k-Nearest Neighbors achieved only 51.34% and 48.50%, respectively.

Performance comparison across different graph types, connectivity schemes, and signal features.

Key Insights

  • Inter-band Connectivity: Linking frequency bands provided the most significant performance gains, as it expanded the network's receptive field across the brain's functional layers.
  • The Sparsity Advantage: Lower-density graphs (fewer edges) often yielded higher accuracy, suggesting that excessive connectivity introduces noise that can obscure meaningful neural patterns.
  • Network Depth: Increasing the number of layers in the GCNN was found to be more effective for improving accuracy than increasing the mathematical complexity of individual filters.
  • Structural Validity: Both distance-based and correlation-based graphs outperformed random baselines, confirming that the underlying biological and functional structure of the brain is essential for effective signal modeling.

Conclusion

This study demonstrates that modeling EEG as a graph allows deep learning models to capture complex patterns that traditional feature vectors miss. While the low dimensionality of EEG (32 channels) remains a limitation, the use of inter-band connectivity and sparsified graphs provides a powerful path forward for identifying mental states and stimuli. Future work will focus on refining edge definitions and exploring the interplay between different signal features to further enhance identification accuracy.

Summary updated

Abstract

This paper proposes a novel graph signal-based deep learning method for electroencephalography (EEG) and its application to EEG-based video identification. We present new methods to effectively represent EEG data as signals on graphs, and learn them using graph convolutional neural networks. Experimental results for video identification using EEG responses obtained while watching videos show the effectiveness of the proposed approach in comparison to existing methods. Effective schemes for graph signal representation of EEG are also discussed.
BibTeX
@inproceedings{jang2018eeg,
  author = {Soobeom Jang and Seong-Eun Moon and Jong-Seok Lee},
  title = {{EEG-based video identification using graph signal modeling and graph convolutional neural network}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2018},
  url = {https://openreview.net/forum?id=nNHTu_hxUjld},
}
Figure