BETA: A Large Benchmark Database Toward SSVEP-BCI Application

Bingchuan Liu, Xiaoshan Huang, Yijun Wang, Xiaogang Chen, Xiaorong GaoFrontiers in Neuroscience 2020 · DOI

TL;DR

This paper introduces BETA, a large-scale benchmark dataset for Steady-State Visual Evoked Potential (SSVEP) Brain-Computer Interfaces (BCIs), featuring 70 subjects and 40 targets in a real-world-like setting. They validate the dataset by comparing eleven SSVEP detection algorithms and propose new metrics for evaluating SSVEP quality and BCI performance.

What Problem Does This Paper Address?

Brain-Computer Interfaces (BCIs) offer a promising way for individuals to communicate and interact with the world, especially those with motor impairments. SSVEP-based BCIs, which leverage brain responses to flickering visual stimuli, have seen significant progress in signal processing and data sharing. However, a major bottleneck has been the limited availability of public datasets, hindering the development and rigorous evaluation of new algorithms. Existing datasets often lack the scale, diversity, or real-world applicability needed to truly advance SSVEP-BCI technology. This paper aims to address this by creating a large, comprehensive, and practically relevant SSVEP dataset.

Methods

The researchers developed the BETA (BEnchmark database Towards BCI Application) dataset. Key aspects of their methodology include:

  • Data Acquisition: They collected 64-channel EEG data from 70 healthy volunteers performing a cued-spelling task using a 40-target virtual keyboard. Crucially, the data was collected outside of a shielded laboratory environment to better reflect real-world conditions.
  • Stimulus Design: A QWERTY-like virtual keyboard was presented on a monitor, with each target flickering at a unique frequency and phase. This "joint frequency and phase modulation" (JFPM) is a standard technique for SSVEP BCIs.
  • Task Procedure: Participants performed a cued-spelling task across four blocks. Trials involved a cue for target selection, followed by stimulus flickering, and a brief rest period for feedback. The flickering duration varied slightly between participants (2s or 3s).
  • Data Preprocessing: Standard EEG preprocessing steps were applied, including band-pass filtering (3-100 Hz), epoch extraction, and down-sampling to 250 Hz.
  • Algorithm Evaluation: Eleven different SSVEP frequency recognition methods (six supervised and five training-free) were implemented and evaluated on the BETA dataset to demonstrate its utility.
  • Metric Development: The paper proposes and validates two metrics:
  • Wide-band Signal-to-Noise Ratio (SNR): A more comprehensive SNR metric that considers harmonics and wide-band noise.
  • BCI Quotient: A normalized score that rescales an individual's SNR to a standard distribution, aiming to provide a population-level measure of BCI performance potential.

Datasets

  • BETA Database (Newly Introduced):
    • Participants: 70 healthy volunteers.
  • Channels: 64-channel EEG.
  • Targets: 40 unique stimuli.
  • Task: Cued-spelling task.
  • Environment: Non-shielded laboratory setting (real-world oriented).
  • Data Format: MATLAB .mat files.
  • Availability: Publicly available at http://bci.med.tsinghua.edu.cn/download.html.
  • Benchmark Database (for comparison): The paper references and compares BETA against a previously published benchmark dataset (Wang et al., 2017) to validate their findings and the utility of BETA.

Key Insights and Ideas

  • Real-world applicability is key: Collecting data outside a shielded room is crucial for developing BCIs that can be used in everyday environments, even if it leads to lower SNR.
  • Large datasets are essential for robust evaluation: The BETA dataset's large number of subjects (70) allows for more reliable estimation of algorithm performance and reduces the risk of overfitting.
  • Wide-band SNR is a better predictor of performance: The study suggests that the wide-band SNR metric is more correlated with Information Transfer Rate (ITR) than the traditional narrow-band SNR, offering a more holistic view of signal quality.
  • BCI Quotient offers a standardized measure: This new metric provides a way to compare individual BCI performance on a population level, analogous to IQ scores, which could be valuable for clinical assessments and user profiling.
  • Algorithm performance varies with data length: Different SSVEP detection algorithms show optimal performance at different epoch lengths, highlighting the importance of considering temporal dynamics in algorithm selection.

Key Results

  • SNR Comparison: The BETA database exhibits significantly lower narrow-band and wide-band SNR compared to the benchmark database (Wang et al., 2017). This is attributed to the non-shielded environment and potentially shorter stimulus durations.
  • Algorithm Performance:
    • Supervised methods: msTRCA and m-Extended CCA generally outperformed other supervised methods, especially for shorter data lengths.
  • Training-free methods: FBCCA showed superior performance for shorter data lengths, while CVARS was better for longer durations.
  • Overall, supervised methods generally achieved higher ITR than training-free methods.
  • SNR vs. ITR Correlation: A strong positive correlation was found between both narrow-band and wide-band SNR and ITR. The wide-band SNR showed a stronger correlation (R²=0.531) compared to narrow-band SNR (R²=0.368).
  • Visual Latency: The estimated visual latency from the BETA dataset (124.96 ± 14.81 ms) was comparable to previous studies.

Limitations and Future Work

  • Lower SNR: The BETA database has a lower SNR compared to some previous datasets, which presents a challenge for existing algorithms and opens opportunities for developing more robust methods.
  • Algorithm Comparison Scope: While eleven algorithms were compared, the authors acknowledge that the performance of some methods might be affected by the specific training data available (e.g., TRCA).
  • Future Work: The authors envision BETA serving as a testbed for:
  • Developing robust frequency recognition algorithms for real-world applications.
  • Studying transfer learning to exploit common patterns across subjects.
  • Investigating few-shot learning approaches due to the reduced number of blocks per subject.
  • Exploring customized paradigms by selecting subsets of the 40 targets.
  • Facilitating population-level brain modeling and developing novel classification approaches, potentially including federated learning.

Summary updated

Abstract

The brain-computer interface (BCI) provides an alternative means to communicate and it has sparked growing interest in the past two decades. Specifically, for Steady-State Visual Evoked Potential (SSVEP) based BCI, marked improvement has been made in the frequency recognition method and data sharing. However, the number of pubic databases is still limited in this field. Therefore, we present a <b>BE</b>nchmark database <b>T</b>owards BCI <b>A</b>pplication (BETA) in the study. The BETA database is composed of 64-channel Electroencephalogram (EEG) data of 70 subjects performing a 40-target cued-spelling task. The design and the acquisition of the BETA are in pursuit of meeting the demand from real-world applications and it can be used as a test-bed for these scenarios. We validate the database by a series of analyses and conduct the classification analysis of eleven frequency recognition methods on BETA. We recommend using the metric of wide-band signal-to-noise ratio (SNR) and BCI quotient to characterize the SSVEP at the single-trial and population levels, respectively. The BETA database can be downloaded from the following link http://bci.med.tsinghua.edu.cn/download.html.
BibTeX
@inproceedings{liu2020beta,
  author = {Bingchuan Liu and Xiaoshan Huang and Yijun Wang and Xiaogang Chen and Xiaorong Gao},
  title = {{BETA: A Large Benchmark Database Toward SSVEP-BCI Application}},
  booktitle = {Frontiers in Neuroscience},
  year = {2020},
  doi = {10.3389/fnins.2020.00627},
}
Figure