Feedback Type May Change the EMG Pattern and Kinematics During Robot Supported Upper Limb Reaching Task

Yasuhiro Kato, Toshiaki Tsuji, Imre CikajloIEEE Open Journal of Engineering in Medicine and Biology 2024 · DOI

TL;DR

This study explores how different types of feedback (visual, haptic, multimodal) influence upper limb reaching movements in a virtual reality environment. They found that terminal haptic feedback significantly improved movement speed and efficiency, and that the size of the target object also affected muscle activation patterns, suggesting that feedback and task context are crucial for tailoring rehabilitation strategies.

What Problem Does This Paper Address?

Robotic systems are increasingly used in rehabilitation to assist with motor learning and recovery. A key aspect of these systems is providing feedback to the user about their performance. However, there's a need to better understand how different feedback modalities (visual, auditory, haptic) and task parameters, like object size, affect a person's movement strategy. Specifically, this paper investigates how concurrent visual feedback combined with various terminal feedback types (given at the end of a movement) influences kinematic parameters (like movement speed and time) and electromyography (EMG) signals, which reflect muscle activity, during a 3D upper limb reaching task. They also explore whether the size of the target object in the virtual environment alters these responses.

Methods

The researchers designed a 3D upper limb reaching task within a virtual reality (VR) environment. Participants used a robot arm as a haptic interface to interact with virtual objects. The setup included:

  • VR Environment: A screen displayed the VR scene, showing the user's hand position (pink ball) and a target object (yellow ball). The target object's size could be adjusted to "big-ball" or "small-ball" modes, influencing the reaching distance. A virtual wall provided boundary feedback.
  • Feedback Modalities:
    • Concurrent Visual Feedback: Always present, showing the target and hand position.
  • Terminal Feedback: Delivered at the end of a successful reach. This included:
  • Terminal Visual Feedback (TV): The target ball changed color upon contact.
  • Terminal Haptic Feedback (TH): The robot provided a sensation of touching the target object.
  • Terminal Multimodal Feedback (TM): A combination of visual and haptic terminal feedback.
  • Kinematic Measurement: The robot's end-effector provided precise tracking of the user's hand movements. Key kinematic variables analyzed were:
  • Movement time
    • Mean velocity
    • Peak velocity
    • Time to peak velocity
    • Velocity peak number (an indicator of smoothness)
  • EMG Measurement: Surface EMG was used to record activity from five muscles in the upper arm (biceps brachii, triceps brachii, upper trapezius, anterior deltoid, posterior deltoid). The primary EMG metric analyzed was peak latency, defined as the time from target appearance to the peak EMG signal.

Datasets

Not discussed in the paper. The study used data collected from 10 healthy, right-handed participants performing the described reaching task.

Key Insights and Ideas

  • Haptic Feedback is Powerful: Terminal haptic feedback, even when combined with visual cues, had a more significant impact on improving movement efficiency (shorter movement time, faster mean velocity) than visual feedback alone. This suggests that the physical sensation of contact is highly informative for motor control.
  • Multimodal Not Always Superior: While multimodal feedback is often assumed to be best, in this study, terminal haptic feedback alone was as effective, and sometimes more so, than multimodal feedback. This could be due to increased cognitive load with more feedback types, potentially hindering learning.
  • Object Size Matters for Muscle Activation: The size of the target object in the VR environment influenced muscle activation patterns, as indicated by differences in EMG peak latency across various muscles. This highlights that even when the target location is the same, changes in target properties (like size) can alter how the body prepares and executes movements.
  • Kinematic Parameters as Sensitive Indicators: The study successfully demonstrated that kinematic parameters like movement time and velocity profiles are sensitive to feedback types and task conditions, making them valuable for assessing movement strategy changes.

Key Results

  • Movement Time: Terminal haptic and multimodal feedback resulted in significantly shorter movement times compared to baseline and terminal visual feedback, particularly in the small-ball mode. For example, movement time was reduced by approximately 0.7 seconds with TH and TM compared to no terminal feedback.
  • Mean Velocity: Mean velocity increased with terminal haptic and multimodal feedback, indicating faster overall movement.
  • Peak Latency: While overall differences in EMG peak latency across feedback types were not statistically significant, there were notable differences based on object size. For instance, the biceps brachii showed smaller peak latency in the small-ball mode, suggesting a different muscle activation strategy depending on the target size.
  • Movement Smoothness: All feedback conditions generally led to smoother movements (fewer velocity peaks) compared to the baseline.

Limitations and Future Work

  • Participant Pool: The study was conducted on 10 healthy participants. The authors suggest that future work should include participants who have experienced a stroke to compare the influence of feedback modalities on movement strategies in different populations.
  • Cognitive Load: The study speculates that the lack of clear superiority of multimodal feedback might be due to increased cognitive load. Further research could investigate this aspect more directly.
  • Task Complexity: The reaching task was 3D, but future studies could explore more complex tasks or activities of daily living.
  • EMG Latency Interpretation: While EMG peak latency showed size-dependent differences, the authors note that it didn't show significant differences based on feedback modality. Further investigation into other EMG parameters or more advanced signal processing might yield deeper insights.

The findings are considered important for designing more effective robotic rehabilitation systems with VR, particularly in tailoring feedback and task parameters to individual needs and conditions.

Summary updated

Abstract

Haptic interfaces and virtual reality (VR) technology have been increasingly introduced in rehabilitation, facilitating the provision of various feedback and task conditions. However, correspondence between the feedback/task conditions and movement strategy during reaching tasks remains a question. To investigate movement strategy, we assessed velocity parameters and peak latency of electromyography. Ten neuromuscularly intact volunteers participated in the measurement using haptic interface and VR. Concurrent visual feedback and various terminal feedback (e.g., visual, haptic, visual and haptic) were given. Additionally, the object size for the reaching task was changed. The results demonstrated terminal haptic feedback had a significant impact on kinematic parameters; showed [Formula: see text] s ([Formula: see text]) shorter movement time and [Formula: see text] m/s ([Formula: see text]) higher mean velocity compared to no terminal feedback. Also, smaller peak latency was observed in different muscle regions based on the object size.
BibTeX
@article{kato2024feedback,
  author = {Yasuhiro Kato and Toshiaki Tsuji and Imre Cikajlo},
  title = {{Feedback Type May Change the EMG Pattern and Kinematics During Robot Supported Upper Limb Reaching Task}},
  journal = {IEEE Open Journal of Engineering in Medicine and Biology},
  year = {2024},
  doi = {10.1109/ojemb.2024.3363137},
}
Figure