Flaws in current human training protocols for spontaneous Brain-Computer Interfaces: lessons learned from instructional design

Fabien Lotte, Florian Larrue, Christian MühlFrontiers in Human Neuroscience 2013 · DOI

TL;DR

Current Brain-Computer Interface (BCI) training protocols, particularly for mental imagery-based BCIs, are largely suboptimal and hinder user learning. By applying principles from instructional design, this paper identifies critical flaws in feedback, instructions, and task design, proposing theoretically grounded improvements to enhance BCI skill acquisition and robustness.

What Problem Does This Paper Address?

Despite the promise of Brain-Computer Interfaces (BCIs) for communication and control, especially for individuals with severe disabilities, their real-world adoption is severely limited by a lack of robustness and reliability. A primary reason for this is the difficulty in accurately and quickly recognizing users' mental states from their brain activity (e.g., EEG). While much research has focused on improving the signal processing algorithms, the authors argue that a crucial, often neglected, component is the human user and how effectively they learn to control the BCI. Spontaneous BCIs, which rely on users generating specific brain patterns through mental imagery, are particularly sensitive to the user's skill. Current training protocols for these BCIs have remained largely unchanged for years and are criticized for being inadequate, leading to low performance and a significant rate of "BCI illiteracy" (users unable to effectively control the system). This paper aims to bridge this gap by analyzing existing BCI training methods through the lens of instructional design principles to identify their shortcomings and suggest evidence-based improvements.

Methods

The paper's methodology is primarily a literature review and theoretical analysis. The authors draw upon established research in instructional design, educational psychology, and human factors to critique current BCI training protocols. They systematically analyze BCI training at three key levels:

  1. Feedback: How users are informed about their performance.
  2. Instructions: The guidance provided to users before and during training.
  3. Tasks: The specific activities users engage in to learn BCI control.

For each level, they identify specific flaws by comparing current BCI practices against established instructional design guidelines. Based on these identified flaws, they propose new research directions and theoretical hypotheses for improving BCI training.

Datasets

Not discussed in the paper. The paper focuses on analyzing existing BCI training protocols and proposing theoretical improvements, rather than introducing new datasets or benchmarks.

Key Insights and Ideas

  • The "Human in the Loop" is Underserved: While signal processing algorithms have seen extensive research, the human user's learning process and the effectiveness of BCI training protocols have been largely overlooked, despite being critical for BCI robustness.
  • Current BCI Training is Suboptimal: Existing training methods for spontaneous BCIs fail to meet many fundamental requirements of effective instructional design, leading to inefficient learning and poor user performance.
  • Feedback Needs a Major Overhaul: Current BCI feedback is often evaluative, non-specific, and lacks explanatory power. Effective feedback should be supportive, meaningful, specific, and clearly indicate how to improve, not just if the task was done correctly.
  • Instructions are Too Basic: BCI instructions often lack clear goals and fail to activate prior user knowledge or provide demonstrations, which are crucial for skill acquisition.
  • Tasks Need to Be More Engaging and Adaptive: Repetitive, synchronous tasks are less effective than adaptive, progressive, and potentially game-like tasks that cater to user motivation and autonomy.
  • "BCI Illiteracy" is a Training Problem: The significant percentage of users who struggle with BCIs might not be inherently incapable, but rather are hindered by poorly designed training.

Key Results

This paper does not present empirical results from new experiments. Instead, it offers a theoretical framework for improving BCI training based on existing research. The "results" are the identified flaws and the proposed, theoretically grounded, suggestions for improvement, which are summarized in Table 1. These include:

  • Feedback: Suggests positive feedback for beginners, disconfirmatory for advanced users, and providing more specific information about EEG patterns (e.g., relevant features, quality measures). Multimodal and engaging feedback environments (like games) are also recommended.
  • Instructions: Advocates for clearly stating the goal of BCI training (producing stable EEG patterns), explaining feedback mechanisms, activating prior user experience, and demonstrating successful BCI use.
  • Tasks: Recommends adaptive training with increasing difficulty, incorporating free/self-paced sessions, using engaging and emotionally positive tasks, introducing task variability, and matching training protocols to individual user characteristics.

Limitations and Future Work

The authors explicitly state that their suggestions are based on theory and require formal validation through empirical studies. They acknowledge that the effectiveness of these proposed improvements needs to be tested to confirm their impact on BCI performance and user learning. Future work should focus on designing and evaluating BCI training protocols that incorporate these instructional design principles. They also suggest that BCI research papers should more thoroughly describe their training protocols to allow for better comparison and understanding of BCI designs.

Summary updated

Abstract

While recent research on Brain-Computer Interfaces (BCI) has highlighted their potential for many applications, they remain barely used outside laboratories. The main reason is their lack of robustness. Indeed, with current BCI, mental state recognition is usually slow and often incorrect. Spontaneous BCI (i.e., mental imagery-based BCI) often rely on mutual learning efforts by the user and the machine, with BCI users learning to produce stable ElectroEncephaloGraphy (EEG) patterns (spontaneous BCI control being widely acknowledged as a skill) while the computer learns to automatically recognize these EEG patterns, using signal processing. Most research so far was focused on signal processing, mostly neglecting the human in the loop. However, how well the user masters the BCI skill is also a key element explaining BCI robustness. Indeed, if the user is not able to produce stable and distinct EEG patterns, then no signal processing algorithm would be able to recognize them. Unfortunately, despite the importance of BCI training protocols, they have been scarcely studied so far, and used mostly unchanged for years. In this paper, we advocate that current human training approaches for spontaneous BCI are most likely inappropriate. We notably study instructional design literature in order to identify the key requirements and guidelines for a successful training procedure that promotes a good and efficient skill learning. This literature study highlights that current spontaneous BCI user training procedures satisfy very few of these requirements and hence are likely to be suboptimal. We therefore identify the flaws in BCI training protocols according to instructional design principles, at several levels: in the instructions provided to the user, in the tasks he/she has to perform, and in the feedback provided. For each level, we propose new research directions that are theoretically expected to address some of these flaws and to help users learn the BCI skill more efficiently.
BibTeX
@inproceedings{lotte2013flaws,
  author = {Fabien Lotte and Florian Larrue and Christian Mühl},
  title = {{Flaws in current human training protocols for spontaneous Brain-Computer Interfaces: lessons learned from instructional design}},
  booktitle = {Frontiers in Human Neuroscience},
  year = {2013},
  doi = {10.3389/fnhum.2013.00568},
}
Figure