Deep Learning of Movement Intent and Reaction Time for EEG-informed Adaptation of Rehabilitation Robots

Neelesh Kumar, Konstantinos P. Michmizos

Abstract

Mounting evidence suggests that adaptation is a crucial mechanism for rehabilitation robots in promoting motor learning. Yet, it is commonly based on robot-derived movement kinematics, which is a rather subjective measurement of performance, especially in the presence of a sensorimotor impairment. Here, we propose a deep convolutional neural network (CNN) that uses electroencephalography (EEG) as an objective measurement of two kinematics components that are typically used to assess motor learning and thereby adaptation: i) the intent to initiate a goal-directed movement, and ii) the reaction time (RT) of that movement. We evaluated our CNN on data acquired from an in-house experiment where 13 subjects moved a rehabilitation robotic arm in four directions on a plane, in response to visual stimuli. Our CNN achieved average test accuracies of 80.08% and 79.82% in a binary classification of the intent (intent vs. no intent) and RT (slow vs. fast), respectively. Our results demonstrate how individual movement components implicated in distinct types of motor learning can be predicted from synchronized EEG data acquired before the start of the movement. Our approach can, therefore, inform robotic adaptation in real-time and has the potential to further improve one's ability to perform the rehabilitation task.

Introduction

Rehabilitation robots are essential tools for restoring motor function through repetitive, "assist-as-needed" therapy. For these systems to be truly effective, they must adapt their assistance levels based on the patient's real-time performance. Traditionally, this adaptation relies on kinematics—measurements like speed and trajectory recorded by the robot's sensors.

However, kinematics can be a misleading metric for patients with sensorimotor impairments. A patient may possess the neural intent to move correctly, but physical limitations may mask this effort in the robotic data. To address this, we propose using electroencephalography (EEG) to provide an objective window into the brain's activity, decoding two critical components of motor learning: Movement Intent (the explicit decision to move) and Reaction Time (a proxy for implicit motor learning).

Methodology

We developed a 5-layer Convolutional Neural Network (CNN) designed to extract spatial and temporal features from EEG data with minimal preprocessing. The system processes 0.5 seconds of EEG data recorded immediately before movement initiation, using 128 channels sampled at 250Hz. Unlike traditional methods that require complex, hand-crafted feature engineering, our approach uses only basic filtering (1-40Hz) and Independent Component Analysis (ICA) to remove artifacts, making it highly suitable for real-time clinical applications.

Experimental setup featuring a subject with a high-density EEG cap operating the robotic arm.

Experimental Design

The study involved 13 healthy subjects performing goal-directed movements using a Bionik InMotion Arm rehabilitation robot. Each subject completed 210 "active" trials (voluntary movement) and 210 "passive" trials (robot-led movement). The CNN was trained to classify these trials into binary categories: Intent vs. No Intent and Fast vs. Slow Reaction Time (based on subject-specific medians).

Results and Analysis

The CNN demonstrated robust performance across different validation schemes, proving that neural signatures of intent and speed are decodable before physical movement begins.

Classification performance metrics, including subject-specific accuracies (A) and confusion matrices for intent and reaction time (B).

Key findings include:

  • Movement Intent: The model achieved an average accuracy of 80.08% in leave-one-out cross-validation, which increased to 88.97% when utilizing subject-specific training data.
  • Reaction Time: The CNN classified movements as "fast" or "slow" with an average accuracy of 79.82% (leave-one-out).
  • Generalization: When trained on a pooled dataset from all subjects, accuracies reached 87.34% for Intent and 84.68% for RT, suggesting the model can generalize across different individuals.

Discussion and Future Work

This research highlights the potential for "dual-path learning" in robotic rehabilitation. By targeting both explicit intent and implicit reaction time, robots can distinguish between a patient who is highly engaged but physically slow and one who is not mentally participating.

While these results in healthy subjects are promising, future work must validate this framework in clinical populations, such as stroke survivors, whose neural signals may be more disorganized. Additionally, moving beyond binary classification to real-time, closed-loop force adjustment will be the next step in creating truly personalized rehabilitation experiences.

Summary updated

Abstract

Mounting evidence suggests that adaptation is a crucial mechanism for rehabilitation robots in promoting motor learning. Yet, it is commonly based on robot-derived movement kinematics, which is a rather subjective measurement of performance, especially in the presence of a sensorimotor impairment. Here, we propose a deep convolutional neural network (CNN) that uses electroencephalography (EEG) as an objective measurement of two kinematics components that are typically used to assess motor learning and thereby adaptation: i) the intent to initiate a goal-directed movement, and ii) the reaction time (RT) of that movement. We evaluated our CNN on data acquired from an in-house experiment where 13 subjects moved a rehabilitation robotic arm in four directions on a plane, in response to visual stimuli. Our CNN achieved average test accuracies of 80.08% and 79.82% in a binary classification of the intent (intent vs. no intent) and RT (slow vs. fast), respectively. Our results demonstrate how individual movement components implicated in distinct types of motor learning can be predicted from synchronized EEG data acquired before the start of the movement. Our approach can, therefore, inform robotic adaptation in real-time and has the potential to further improve one's ability to perform the rehabilitation task.
BibTeX
@inproceedings{kumar2020deep,
  author = {Neelesh Kumar and Konstantinos P. Michmizos},
  title = {{Deep Learning of Movement Intent and Reaction Time for EEG-informed Adaptation of Rehabilitation Robots}},
  booktitle = {BioRob},
  year = {2020},
  url = {https://openreview.net/forum?id=UzP8qkxCg7},
}
Figure