TL;DR
This study investigates how different feedback modalities (visual vs. visual + vibrotactile) and their presentation order impact motor imagery-based Brain-Computer Interface (MI-BCI) user training over multiple sessions. The findings reveal that while multimodal feedback generally improves performance, starting with visual feedback first leads to better initial results and user motivation, suggesting a strategic approach to feedback introduction is crucial for effective MI-BCI training.
What Problem Does This Paper Address?
Motor-Imagery based Brain-Computer Interfaces (MI-BCIs) hold great promise for controlling digital technologies and aiding in neurorehabilitation. However, their widespread adoption is hindered by low performance and the need for extensive user training. A critical component of this training is the feedback provided to the user, which informs them about their brain activity and the system's interpretation. While previous research suggests multimodal feedback (combining visual and other senses) is superior to purely visual feedback for short-term training, the long-term, multi-session effects of such feedback, and crucially, the impact of the order in which different feedback modalities are introduced, remained largely unexplored. This paper aims to fill this gap by systematically evaluating the influence of both feedback modality and its presentation sequence on MI-BCI user performance and experience over several training sessions.
Methods
The study employed a within-participant design where 16 naive MI-BCI participants underwent 10 training sessions. Each participant experienced both a unimodal visual feedback condition and a multimodal visual + vibrotactile feedback condition, with the order of these conditions randomized across participants. This allowed for a direct comparison of the feedback modalities and the effect of their presentation sequence.
Key aspects of the methodology include:
- Feedback Modalities:
- Unimodal Visual Feedback: A realistic visual representation of arms performing hand movements, aiming to induce a sense of embodiment.
- Multimodal Feedback: The same realistic visual feedback, augmented with vibrotactile feedback delivered to the wrist via motors in gloves. The intensity of vibration was proportional to the classifier's confidence.
- Training Protocol: Participants performed motor imagery tasks (imagining hand movements) over 10 sessions. Sessions were structured to include baseline recordings, classifier calibration, and task performance with feedback.
- Data Collection:
- MI-BCI Performance: Measured by the online classification accuracy of motor imagery tasks.
- User Experience: Assessed using the NeXT questionnaire, measuring cognitive load and agency, and through self-reported changes in mood, mindfulness, and motivation between the start and end of sessions.
- Participant Profiling: Questionnaires were used to assess personality traits (autonomy, tension), mental rotation abilities, and initial visual and kinaesthetic imagery abilities.
- Statistical Analysis: 3-way repeated measures ANOVAs were used to analyze the influence of "Modality," "Order," and "Session" on MI-BCI performance and user experience indicators. Correlation analyses were performed to investigate the impact of participant profiles.
Datasets
Not explicitly discussed as separate datasets. The study utilized data collected from the 16 participants during their MI-BCI training sessions for analysis.
Key Insights and Ideas
- Multimodal Advantage: The study confirms that multimodal feedback (visual + vibrotactile) generally leads to higher MI-BCI classification accuracy compared to unimodal visual feedback, reinforcing previous findings.
- Order Matters: A significant finding is that the order of feedback presentation profoundly impacts MI-BCI performance. Participants who started with unimodal visual feedback achieved higher performance than those who began with multimodal feedback.
- Novice Overwhelm: The authors hypothesize that integrating information from two modalities simultaneously might be overwhelming for novice learners, suggesting a phased approach to feedback introduction is beneficial.
- User Experience Fluctuations: While multimodal feedback showed initial advantages in user experience (e.g., higher agency in early sessions), the visual-only feedback seemed to lead to better sustained motivation and positive mood/mindfulness over time.
- No Strong Predictors from Profile: Interestingly, basic participant profiling (autonomy, tension, mental rotation, initial kinaesthetic imagery) did not strongly correlate with MI-BCI performance in this study, though initial visual imagery abilities showed a difference between groups, which was controlled for.
Key Results
- MI-BCI Performance:
- Multimodal feedback led to significantly higher MI classification accuracy (M = 69.61%) compared to visual feedback alone (M = 66.66%).
- Participants who started with visual feedback achieved significantly higher overall MI classification accuracy (M = 72.95%) than those who started with multimodal feedback (M = 63.32%).
- User Experience:
- Mindfulness: Multimodal feedback had a significant influence on mindfulness, with a greater decrease observed when training with visual feedback only.
- Agency: The interaction between "Modality" and "Session" significantly influenced sense of agency, with multimodal feedback initially leading to higher agency, but this trend reversed in later sessions.
- Mood: Similar to agency, "Modality*Session" interaction affected mood, with multimodal feedback showing initial benefits.
- Motivation: The "Session*Order" interaction significantly influenced motivation, with participants starting with visual feedback showing less demotivation.
Limitations and Future Work
- Participant Number: The study included 16 participants, which the authors acknowledge is a limitation. Future research with a larger sample size is recommended to confirm the findings.
- Number of Sessions per Modality: Participants trained with each modality for only five sessions. More extensive training with each modality could provide deeper insights into multi-session effects.
- Feedback Congruence: The study aimed for congruence between virtual and real arms, but temporal and spatial asynchronies could have impacted the sense of agency, particularly for low performers.
- Motor Execution vs. Imagery: The inclusion of motor execution tasks alongside motor imagery tasks might have influenced results due to shared neural pathways.
- Future Work: The authors suggest exploring proprioceptive feedback as an alternative to vibrotactile feedback and investigating the impact of different feedback strategies for users with varying skill levels. Further research is also needed to understand the influence of initial visual and kinaesthetic imagery abilities on BCI performance, especially considering different task types and feedback modalities.
Summary updated