TL;DR
This study investigates how auditory stimuli, specifically the speed of an auditory story, impact the performance of visual Brain-Computer Interfaces (BCIs) that rely on Event-Related Potentials (ERPs). The findings reveal that increased auditory speed elevates mental workload, leading to reduced ERP amplitudes, lower accuracy, and decreased information transfer rates in visual BCIs, highlighting the importance of considering environmental factors for real-world BCI applications.
What Problem Does This Paper Address?
Brain-Computer Interfaces (BCIs) that utilize Event-Related Potentials (ERPs) are promising for assisting individuals with motor impairments. However, their performance often degrades when transitioning from controlled laboratory settings to real-world environments. A significant factor in real-world environments is the presence of ambient sound, which can act as a distractor or cognitive load. While some studies have explored the effects of sound on auditory BCIs, the impact of sound on visual ERP-based BCIs, particularly how it influences mental workload and consequently performance, remains less understood. This paper aims to bridge this gap by systematically examining how varying auditory task difficulty (through speech speed) affects visual BCI performance.
Methods
The researchers designed a novel dual-task experiment where participants simultaneously engaged in a visual BCI task and an auditory task. The visual task involved identifying "target" robotic arm images within a matrix using an oddball paradigm, which elicits ERPs like P300 and N200. The auditory task consisted of listening to an audio story at three different speeds: no sound (0-T, low workload), half speed (0.5-T, medium workload), and normal speed (1-T, high workload). Participants were instructed to silently count specific words ("De") in the story while performing the visual BCI task.
Mental workload was assessed using the NASA-TLX scale. EEG data was collected and pre-processed, including filtering and baseline correction. ERPs were extracted and analyzed for amplitude and latency changes. Classification was performed using Fisher Linear Discriminant Analysis (FLDA). The study evaluated performance based on classification accuracy and Information Transfer Rate (ITR), and importantly, analyzed the impact of training and testing classifiers with data from the same or different workload conditions.
Datasets
The study utilized EEG data collected from 10 participants performing the described visual-auditory dual-task experiment. No pre-existing public datasets were used for training or evaluation. The experiment generated a novel dataset specific to the visual-auditory BCI paradigm under varying auditory workloads.
Key Insights and Ideas
- Auditory Speed as a Workload Manipulator: The study effectively demonstrates that the speed of an auditory task can directly influence mental workload, impacting cognitive resource allocation.
- Dual-Pathway Interaction: The research highlights the interconnectedness of auditory and visual pathways in the brain when performing a BCI task, suggesting that auditory input can interfere with visual processing.
- ERP Amplitude as a Workload Indicator: Increased auditory workload leads to a significant decrease in P300 and N200 amplitudes, particularly in the occipital-parietal regions, serving as an objective measure of cognitive load.
- Training-Testing Data Similarity is Crucial: Beyond ERP amplitude, the congruence between training and testing data (i.e., using data from similar workload conditions) is paramount for achieving high BCI performance. Mismatched data significantly degrades accuracy.
Key Results
- ERP Amplitude Reduction: Increasing auditory task speed led to a decrease in P300 amplitude by 0.86 µV (p=0.0239) and N200 amplitude by 0.69 µV (p=0.0158) in the occipital-parietal area.
- Performance Decline: This ERP amplitude reduction resulted in a 5.95% decline in accuracy (p=0.0101) and a 9.53 bits/min decline in Information Transfer Rate (ITR) (p=0.0416) as auditory speed increased.
- Workload Impact: The study found that higher workload conditions (HD) generally led to lower accuracy and ITR compared to lower workload conditions (LD).
- Training-Testing Match: Classifiers trained and tested on data from the same workload condition (e.g., LD-LD) consistently outperformed those trained and tested on different conditions (e.g., LD-MD, LD-HD), even if the ERP amplitudes were lower. For instance, LD-LD achieved 90.83% accuracy, while LD-HD achieved 76.39%.
Limitations and Future Work
The authors acknowledge that the study focused on a specific type of auditory stimulus (a story) and a particular visual BCI paradigm. Future work could explore the effects of different types of sounds (e.g., music, environmental noise) and other visual BCI paradigms. The study also noted that the sound of the robotic arm servos during online experiments might have had an additional effect, which warrants further investigation. Additionally, exploring how user familiarity with sounds or memory cognitive mechanisms might influence BCI performance is suggested for future research. The authors also propose investigating how to incorporate environmental noise into the training process to create more robust BCIs for real-world use.
Summary updated