Self-Paced Brain-Computer Interaction with Virtual Worlds: A Quantitative and Qualitative Study "Out of the Lab"

Fabien Lotte, Yann Renard, Anatole Lécuyer2008

TL;DR

This paper explores the feasibility of a "real-world" brain-computer interface (BCI) using a single EEG channel to control a virtual reality game, "Use the Force," inspired by Star Wars. Even without training or machine learning, about half of naive users could control the game with real foot movements, and a quarter with imagined movements, highlighting the potential of accessible BCI systems.

What Problem Does This Paper Address?

Traditional BCI research often takes place in highly controlled laboratory settings with trained users and sophisticated algorithms. This paper aims to bridge the gap between these controlled studies and real-world BCI applications by evaluating a self-paced BCI system in a more challenging, "out-of-the-lab" environment. Specifically, they address the lack of studies on naive users, the need for BCI systems that don't require extensive training or machine learning for adaptation, and the importance of user preferences and experience beyond just performance metrics. They also investigate the effectiveness of a single EEG channel for BCI control, making it more accessible.

Methods

The researchers developed a self-paced (asynchronous) BCI system that detects Beta Event-Related Synchronization (ERS) in the EEG signal, which is associated with motor imagery or real foot movements. The system uses a single EEG channel (Cz) and a simple threshold-based detection method, avoiding complex machine learning. The BCI controls a virtual reality application called "Use the Force," where users attempt to lift a virtual Tie-Fighter spaceship by imagining or performing foot movements. The experiment involved 21 naive subjects who participated in a 45-minute session that included electrode setup, signal visualization, baseline recording, free interaction time, a real movement game, an imagined movement game, and a subjective questionnaire. Performance was evaluated using True Positives (TP) and False Positives (FP), and the Hit-False (HF) difference was calculated.

Datasets

The study used data collected from 21 naive subjects during the Laval Virtual 2008 VR exhibition. No pre-existing datasets were used for training or evaluation. The experiment itself can be considered a newly introduced benchmark for evaluating self-paced BCIs in a challenging, "out-of-the-lab" setting with naive users.

Key Insights and Ideas

  • "Out-of-the-Lab" Feasibility: The study demonstrates that even with a single EEG channel, no training, and a simple detection method, a BCI can achieve meaningful control in a noisy, real-world environment.
  • Naive User Performance: A significant portion of naive users could achieve some level of control, suggesting that BCI technology is becoming more accessible and intuitive.
  • Real vs. Imagined Movements: Real foot movements yielded better performance than imagined movements, which is expected given the direct neural activation. However, the fact that a quarter of users could control the system with imagined movements is promising.
  • Importance of Feedback: Subjective feedback strongly emphasized the need for continuous and complete visual feedback, even when the system is not actively controlled. Lack of feedback led to increased frustration.
  • Enjoyment and Motivation: The VR application was found to be enjoyable and motivating, highlighting the potential of gamified BCI experiences.

Key Results

  • Performance:
    • Approximately half of the subjects (12 out of 21) achieved an HF difference ≥ 3 using real movements, indicating a performance better than random.
  • About a quarter of the subjects (5 out of 21) achieved an HF difference ≥ 3 using imagined movements.
  • Mean HF difference was 3.14±2.24 for real movements and 1.33 ± 2.03 for imagined movements.
  • Mean TP was 4.95 ± 2.18 for real movements and 2.67±2.08 for imagined movements.
  • Qualitative Findings:
    • Subjects found the experiment comfortable (average mark of 5.19/7).
  • Control was perceived as better with real movements (3.95/7) compared to imagined movements (2.81/7).
  • Imagined movement games were perceived as more frustrating and annoying (3.29/7) than real movement games (2.33/7).
  • Subjects reported a wide variety of mental strategies for imagined movements, with those imagining the same movement as the real one performing slightly better.
  • Many subjects found the application well-designed, enjoyable, and motivating.

Limitations and Future Work

  • Limitations:
    • The study was conducted in a noisy exhibition environment, which could have impacted performance.
  • The BCI system used a single EEG channel and a simple threshold, which may limit its potential.
  • The performance results, while promising, are modest, especially for imagined movements.
  • The lack of a machine learning component for adaptation means the system's performance might be suboptimal for some users.
  • Future Work:
    • Investigate the use of different frequency bands and electrode locations for improved performance.
  • Explore more sophisticated BCI designs, potentially incorporating machine learning for user-specific adaptation.
  • Further research into the impact of continuous and varied feedback mechanisms on user frustration and learning.
  • Investigate the effectiveness of specific mental strategies for imagined movements.

Summary updated

Abstract

This paper describes the results of an evaluation of a self-paced BCI application conducted with 21 naïve subjects. We studied both performances and preferences of subjects placed voluntarily in a challenging situation: first-time session, no human learning, no machine learning of the mental state to be detected, “out of the lab”, use of a single EEG channel. The application consisted of an entertaining interaction with a virtual world inspired by the famous “Star Wars ” movie. Subjects were asked to control the take-off and height of a virtual spaceship by using their motor-related brain activity. Results showed that, without training, roughly half of the subjects were able to control the application by using real foot movements and a quarter were able to control it by using imagined foot movements. Taken together, the results of subjective questionnaires stressed the importance of mental strategies and the need to provide subjects with a continuous and complete visual feedback, even when the non-control state is detected. In addition, the whole application appeared as enjoyable and motivating for the subjects. 1
BibTeX
@misc{lotte2008self,
  author = {Fabien Lotte and Yann Renard and Anatole Lécuyer},
  title = {{Self-Paced Brain-Computer Interaction with Virtual Worlds: A Quantitative and Qualitative Study "Out of the Lab"}},
  year = {2008},
}
Figure