Cybathlon experiences of the Graz BCI racing team Mirage91 in the brain-computer interface discipline

Karina Statthaler, Andreas Schwarz, David Steyrl, Reinmar J. Kobler, Maria Katharina Höller, Julia Brandstetter, Lea Hehenberger, Marvin Bigga, Gernot Müller-PutzJournal of NeuroEngineering and Rehabilitation 2017 · DOI

TL;DR

This paper details the journey of the Graz BCI Racing Team MIRAGE91 in the inaugural Cybathlon's Brain-Computer Interface (BCI) race. They successfully developed and trained a BCI system for a pilot with severe motor impairments to play a computer game, highlighting the challenges and rewards of translating BCI research into a competitive, real-world application. The experience underscored the critical need for BCI systems that are robust to environmental and physiological changes, a key hurdle for widespread adoption.

What Problem Does This Paper Address?

The Cybathlon aimed to showcase assistive technologies for individuals with severe motor impairments, pushing the boundaries of what's possible with current BCI technology. The specific challenge for the BCI racing discipline was to enable a pilot to control a virtual avatar in a game using only their thoughts. This requires not only a BCI system capable of reliably decoding mental commands but also a pilot who can consistently generate distinct brain patterns and adapt to the demands of a competition environment. A significant hurdle in BCI research is the variability of brain signals due to factors like fatigue, attention, and environmental changes, which can drastically impact performance, especially in high-stakes situations like a competition.

Methods

The team employed a multi-stage approach to develop and train their BCI system:

  • Pre-screening: Assessed the pilot's ability to generate distinct brain patterns (sensorimotor rhythms - SMR) and their compliance with instructions.
  • Screening: Evaluated seven different mental tasks and a rest condition to identify the optimal four-class combination for the pilot, aiming for high accuracy and comfort. They used Common Spatial Patterns (CSP) filters and a shrinkage Linear Discriminant Analysis (sLDA) classifier.
  • Online BCI with Feedback: Introduced the pilot to a closed-loop BCI system where they received visual feedback on their performance, allowing them to adapt their mental strategies. This stage also involved adapting the BCI system to real-time processing.
  • BCI Game Training: Transitioned to training with the actual "BrainRunners" game. This involved developing a co-adaptive training paradigm and implementing real-time artifact detection (eye blinks, general EEG deviations) to maintain performance during gameplay. They also incorporated a novel approach of using a portion of previous training data to reduce calibration time.

Datasets

The paper does not explicitly mention publicly available datasets. Instead, the team generated their own datasets through:

  • Pre-screening sessions: Collected EEG data using a 16-channel setup.
  • Screening sessions: Expanded to a 32-channel EEG setup to accommodate more complex mental tasks.
  • Online BCI sessions: Collected data for calibration and real-time performance evaluation.
  • Game training sessions: Generated extensive EEG data during gameplay, both in a controlled environment and during the competition.

Key Insights and Ideas

  • The "Rest" Condition is Tricky: The pilot found the "rest" condition (no mental action) uncomfortable and difficult to maintain focus on, leading to a decision to replace it with auditory imagery. This highlights the importance of pilot comfort and preference in task selection for BCI.
  • Environment Matters: The BCI system's performance dropped significantly in the competition arena compared to training sessions. This suggests that environmental factors (noise, spectators, pressure) and the transition from a controlled training setup to a live competition are major challenges for BCI robustness.
  • Co-adaptation is Crucial: The paper emphasizes the need for co-adaptive BCI systems that can adjust to slowly varying changes in EEG and the environment, rather than relying on static training.
  • Pilot Training is Key: The team's rigorous, multi-month training program, including adapting the task set and increasing session frequency, was essential for achieving even their competition performance.

Key Results

  • Training Improvement: The median runtime for the pilot improved significantly over the training period, decreasing from an initial average of 178 seconds to 120 seconds in later stages.
  • Competition Performance Drop: Despite strong training results, the pilot's performance in the actual Cybathlon race was significantly worse, with a runtime of 196 seconds, leading to them not qualifying for the finals.
  • Feature Distribution Shift: Analysis revealed a substantial shift in the feature distribution between training data and data collected during the competition, particularly in the arena, leading to a strong bias towards the "FEET" class (85% of the time).
  • Accuracy: Pre-screening and screening sessions showed accuracies exceeding chance levels, with the best 4-task combination achieving a peak accuracy of 75.6% offline. Online performance during training peaked at 63.1%.

Limitations and Future Work

  • BCI Robustness: The primary limitation identified is the BCI system's lack of robustness to non-task-related changes in EEG, particularly those induced by environmental shifts and physiological states like arousal.
  • Transition Challenges: The transition from a controlled calibration environment to the dynamic game environment, especially with feedback, proved challenging and led to performance degradation.
  • Future Work: The authors suggest several avenues for future research:
  • Developing more adaptive training paradigms that mimic competition environments.
  • Implementing co-adaptive BCI systems that continuously adjust to changing conditions.
  • Investigating the role of physiological states (like arousal) and monitoring them to improve BCI inference.
  • Extending pilot training to include more realistic, high-pressure scenarios.
  • Improving the BCI system's signal processing, stability, and artifact handling.

Summary updated

Abstract

The CYBATHLON 2016 was a great opportunity for our student team. We consolidated our theoretical knowledge and turned it into practice, allowing our pilot to play a computer game. However, further research is required to make BCI technology invariant to non-task related changes of the EEG.
BibTeX
@article{statthaler2017cybathlon,
  author = {Karina Statthaler and Andreas Schwarz and David Steyrl and Reinmar J. Kobler and Maria Katharina Höller and Julia Brandstetter and Lea Hehenberger and Marvin Bigga and Gernot Müller-Putz},
  title = {{Cybathlon experiences of the Graz BCI racing team Mirage91 in the brain-computer interface discipline}},
  journal = {Journal of NeuroEngineering and Rehabilitation},
  year = {2017},
  doi = {10.1186/s12984-017-0344-9},
}
Figure