Training in Use of Brain–Machine Interface-Controlled Robotic Hand Improves Accuracy Decoding Two Types of Hand Movements

Ryohei Fukuma, Takufumi Yanagisawa, Hiroshi Yokoi, Masayuki Hirata, Toshiki Yoshimine, Youichi Saitoh, Yukiyasu Kamitani, Haruhiko KishimaFrontiers in Neuroscience 2018 · DOI

TL;DR

This study demonstrates that training with a real-time brain-machine interface (BMI) controlling a robotic hand significantly improves both the accuracy of robotic control and the brain's ability to decode hand movements. This suggests that BMI training can induce beneficial neuroplasticity, making it a promising tool for rehabilitation and pre-operative evaluation for invasive BMI applications.

What Problem Does This Paper Address?

Brain-machine interfaces (BMIs) hold immense potential for restoring motor function in individuals with paralysis. However, the effectiveness of BMIs can vary significantly between individuals, and the underlying neural mechanisms driving successful control are not fully understood. Specifically, it's unclear whether actively using a BMI to control an external device can lead to improvements in the brain's own representation of movements, thereby enhancing future decoding accuracy. This paper investigates whether training with a real-time BMI, which provides neurofeedback through robotic hand control, can induce plastic changes in the brain that improve the decoding of hand movements.

Methods

The researchers developed a non-invasive BMI system using magnetoencephalography (MEG) to decode hand movements. Healthy participants underwent a series of experiments:

  • Offline Task (Pre-BMI): Participants were instructed to grasp or open their right hand in response to visual cues while their brain activity was recorded using MEG. This data was used to train a "real" decoder.
  • BMI Training (Closed-Loop): Participants then used the real-time BMI to control a robotic hand for 10 minutes. They received visual feedback of the robotic hand's movements, aiming to improve their control. This training was conducted using either a "real" decoder (trained on their own movement data) or a "sham" decoder (trained on shuffled movement labels) in a crossover design to control for placebo effects.
  • Offline Task (Post-BMI): After the training period, participants repeated the initial offline task to assess any changes in their brain's ability to represent hand movements.

Cortical activity was analyzed by estimating cortical currents using a source localization technique (VBMEG). Support Vector Machines (SVMs) were used to decode hand movements from MEG signals, and classification accuracy was compared between pre- and post-training offline tasks.

Datasets

The study used MEG data recorded from nine healthy participants during specific hand movement tasks (grasping and opening). This data was collected in the researchers' lab at Osaka University. No new public datasets were introduced.

Key Insights and Ideas

  • Neurofeedback drives plasticity: The core idea is that actively using a BMI to control an external device (the robotic hand) and receiving feedback about that control can actively reshape the brain's neural representations of movement.
  • Real vs. Sham Decoder is Crucial: The comparison between a "real" decoder (based on actual movement data) and a "sham" decoder (using randomized data) is critical. It helps isolate the effects of genuine BMI control and feedback from mere participation in a training session.
  • BMI Training Enhances Decoding: The study provides evidence that this active training process doesn't just improve the immediate control of the robotic hand but also leads to more robust and accurate decoding of hand movements from brain signals in a subsequent offline task.
  • Potential for Clinical Translation: The findings suggest that this non-invasive BMI training could be a valuable tool for preparing patients for more invasive BMI procedures or for rehabilitation after neurological injury.

Key Results

  • Improved Robotic Hand Control: During the 10-minute BMI training with the real decoder, participants significantly improved their accuracy in controlling the robotic hand, with the correct rate increasing from 0.28 ± 0.13 to 0.50 ± 0.11 (p = 0.017). In contrast, training with the sham decoder showed no significant improvement (0.51 ± 0.13 to 0.52 ± 0.10, p = 0.92).
  • Enhanced Offline Decoding Accuracy: Following BMI training with the real decoder, the accuracy of decoding hand movements during the offline task significantly increased from 62.7 ± 6.5% to 70.0 ± 11.1% (p = 0.022). Training with the sham decoder did not lead to a significant improvement in offline decoding accuracy (63.0 ± 8.8% to 66.4 ± 9.0%, p = 0.225).
  • Cortical Representation Changes: While not statistically significant after Bonferroni correction, there was an observed trend of increased F-values in the contralateral sensorimotor cortex after real BMI training, suggesting changes in the discriminability of cortical activity related to hand movements.

Limitations and Future Work

  • Limited Sample Size: The study involved only nine healthy participants, which is a relatively small sample size.
  • Healthy Participants: The experiments were conducted on healthy individuals. Further research is needed to confirm these findings in patient populations with motor impairments.
  • Short Training Duration: The BMI training lasted only 10 minutes. Longer training periods might yield even greater improvements.
  • Future Directions: The authors suggest that this non-invasive BMI training could be beneficial for:
  • Preoperative Evaluation: Assessing and improving a patient's ability to control a BMI before invasive surgery.
  • Clinical Therapy: Potentially changing maladapted cortical representations in conditions like chronic pain or after stroke.
  • Optimizing BMI Performance: Further research is needed to optimize the training protocols and understand how different cortical activity properties influence performance improvements.

Summary updated

Abstract

<b>Objective:</b> Brain-machine interfaces (BMIs) are useful for inducing plastic changes in cortical representation. A BMI first decodes hand movements using cortical signals and then converts the decoded information into movements of a robotic hand. By using the BMI robotic hand, the cortical representation decoded by the BMI is modulated to improve decoding accuracy. We developed a BMI based on real-time magnetoencephalography (MEG) signals to control a robotic hand using decoded hand movements. Subjects were trained to use the BMI robotic hand freely for 10 min to evaluate plastic changes in the cortical representation due to the training. <b>Method:</b> We trained nine young healthy subjects with normal motor function. In open-loop conditions, they were instructed to grasp or open their right hands during MEG recording. Time-averaged MEG signals were then used to train a real decoder to control the robotic arm in real time. Then, subjects were instructed to control the BMI-controlled robotic hand by moving their right hands for 10 min while watching the robot's movement. During this closed-loop session, subjects tried to improve their ability to control the robot. Finally, subjects performed the same offline task to compare cortical activities related to the hand movements. As a control, we used a random decoder trained by the MEG signals with shuffled movement labels. We performed the same experiments with the random decoder as a crossover trial. To evaluate the cortical representation, cortical currents were estimated using a source localization technique. Hand movements were also decoded by a support vector machine using the MEG signals during the offline task. The classification accuracy of the movements was compared among offline tasks. <b>Results:</b> During the BMI training with the real decoder, the subjects succeeded in improving their accuracy in controlling the BMI robotic hand with correct rates of 0.28 ± 0.13 to 0.50 ± 0.11 (<i>p</i> = 0.017, <i>n</i> = 8, paired Student's <i>t</i>-test). Moreover, the classification accuracy of hand movements during the offline task was significantly increased after BMI training with the real decoder from 62.7 ± 6.5 to 70.0 ± 11.1% (<i>p</i> = 0.022, <i>n</i> = 8, <i>t</i><sub>(7)</sub> = 2.93, paired Student's <i>t-</i>test), whereas accuracy did not significantly change after BMI training with the random decoder from 63.0 ± 8.8 to 66.4 ± 9.0% (<i>p</i> = 0.225, <i>n</i> = 8, <i>t</i><sub>(7)</sub> = 1.33). <b>Conclusion:</b> BMI training is a useful tool to train the cortical activity necessary for BMI control and to induce some plastic changes in the activity.
BibTeX
@inproceedings{fukuma2018training,
  author = {Ryohei Fukuma and Takufumi Yanagisawa and Hiroshi Yokoi and Masayuki Hirata and Toshiki Yoshimine and Youichi Saitoh and Yukiyasu Kamitani and Haruhiko Kishima},
  title = {{Training in Use of Brain–Machine Interface-Controlled Robotic Hand Improves Accuracy Decoding Two Types of Hand Movements}},
  booktitle = {Frontiers in Neuroscience},
  year = {2018},
  doi = {10.3389/fnins.2018.00478},
}
Figure