A Prosthetic Hand Body Area Controller Based on Efficient Pattern Recognition Control Strategies

Simone Benatti, Bojan Milosevic, Elisabetta Farella, Emanuele Gruppioni, Luca Benini

TL;DR

This paper presents a wearable Body Sensor Network (BSN) for controlling prosthetic hands using intuitive EMG-based gesture recognition. By combining an efficient embedded SVM classifier with a novel control strategy, the system achieves a low end-to-end error rate of 1.6% for common hand gestures, offering a more natural and robust interaction for amputees.

What Problem Does This Paper Address?

Modern prosthetic hands offer advanced functionality, but their control interfaces are often non-intuitive. Current systems rely on complex encoding of muscle contractions, requiring extensive training and cognitive effort from users. While myoelectric gesture recognition shows promise for more natural control, there's a significant gap between research-level accuracy and practical, real-time implementation in wearable systems. This paper tackles the challenge of integrating an efficient embedded gesture recognition algorithm with a control strategy that prioritizes intuitive user interaction and robust prosthesis actuation, aiming to bridge this gap for upper-limb amputees.

Methods

The core of the system is a wearable node featuring an ARM Cortex M4 microcontroller. This node handles real-time EMG signal acquisition from four sensors, embedded SVM-based gesture classification, and direct control of the prosthetic hand. The SVM algorithm, specifically using a Radial Basis Function (RBF) kernel, is trained offline on a PC and then deployed to the microcontroller.

A key innovation is the control strategy, which moves beyond simple pattern recognition. It incorporates a Finite State Machine (FSM) that manages transitions between gestures and a "reset" state (open hand). To improve robustness against signal noise and transient phases during gesture onset, a majority voting approach is used over 20 consecutive SVM classifications. This ensures a timely and accurate decision for prosthesis actuation. The system also leverages a smartphone as a personal gateway for training, personalization, and updating the recognition models, enabling cloud-assisted capabilities and periodic retraining without real-time constraints.

Datasets

  • NINAPRO Database: Used for offline validation and comparison. This is a large, publicly available dataset containing EMG signals from 27 subjects performing up to 52 hand gestures, recorded with 10 EMG sensors and a hand-tracking glove. The authors selected a subset of gestures for their study and interleaved them with "open hand" gestures to simulate real-world usage.
  • Custom Dataset: Collected from 5 healthy subjects and 4 transradial amputees during real-time experiments. This dataset was used to evaluate the end-to-end performance of the integrated system in a practical setting.

Key Insights and Ideas

  • Beyond Classification Accuracy: The paper emphasizes that for prosthetic control, end-to-end performance (timely and accurate actuation) is more critical than just classification accuracy. A system that correctly classifies a gesture but acts too late or incorrectly is less useful.
  • Intuitive Control Strategy is Crucial: Simply recognizing gestures isn't enough. The FSM-based control strategy, which includes a mandatory reset state and handles decontraction, significantly improves usability and reduces user fatigue by not requiring continuous muscle contraction to maintain a grasp.
  • Majority Voting for Robustness: Employing majority voting over multiple SVM classifications helps to overcome the transient phase of EMG signals at gesture onset, leading to more reliable gesture detection and reducing misclassifications.
  • Gateway for Personalization: The use of a smartphone gateway for training and model updates is a practical approach that offloads computationally intensive tasks from the wearable device, allowing for personalized control and easier system adaptation.
  • Reduced Sensor Count is Feasible: The study demonstrates that using only 4 EMG sensors can yield comparable results to systems using more sensors (like the 10 used in the NINAPRO dataset), suggesting a path towards more cost-effective and less intrusive prosthetic devices.

Key Results

  • End-to-End Error Rate: The proposed system achieved an average end-to-end error rate of 1.6% (equivalent to 1 misclassified gesture every 60) for common hand gestures in real-time experiments with both healthy subjects and amputees.
  • Recognition Time: The SVM prediction and control FSM achieved a recognition time of 36 ms in the worst case, well within the requirements for real-time control.
  • Memory Footprint: The SVM models, even with a higher number of Support Vectors (SVs) due to fewer input features (4 channels), occupied up to 12 kB, which is manageable for the target microcontroller with 64 kB of Flash memory.
  • Computational Load: The proposed approach with 4 EMG channels demonstrated a reduced computation time compared to the NINAPRO setup using 10 channels, despite potentially having more SVs.
  • Accuracy Comparison: The system showed comparable accuracy to state-of-the-art approaches when tested on the NINAPRO dataset, with an average accuracy of 91.83% for the complete dataset.

Limitations and Future Work

  • Limitations:
    • The paper notes that for amputee patients, the compromised residual muscles can lead to more confused activation patterns, requiring more SVs and increasing computation time for the classifier.
  • While the system was tested on a small number of amputees (4), a larger cohort would provide more robust validation.
  • The "open hand" gesture was sometimes excluded from training to reduce SVs and computation, relying on decontraction to trigger the reset. While effective, this might be less intuitive for some users or specific scenarios.
  • Future Work:
    • The authors suggest consolidating an advanced prototype to enable a detailed evaluation of usability in real-life scenarios.
  • Further comparison with state-of-the-art (SoA) solutions is also proposed.
  • Future work could explore more advanced deep learning techniques for gesture recognition, though the current focus is on efficient embedded implementation.

Summary updated

Abstract

Poliarticulated prosthetic hands represent a powerful tool to restore functionality and improve quality of life for upper limb amputees. Such devices offer, on the same wearable node, sensing and actuation capabilities, which are not equally supported by natural interaction and control strategies. The control in state-of-the-art solutions is still performed mainly through complex encoding of gestures in bursts of contractions of the residual forearm muscles, resulting in a non-intuitive Human-Machine Interface (HMI). Recent research efforts explore the use of myoelectric gesture recognition for innovative interaction solutions, however there persists a considerable gap between research evaluation and implementation into successful complete systems. In this paper, we present the design of a wearable prosthetic hand controller, based on intuitive gesture recognition and a custom control strategy. The wearable node directly actuates a poliarticulated hand and wirelessly interacts with a personal gateway (i.e., a smartphone) for the training and personalization of the recognition algorithm. Through the whole system development, we address the challenge of integrating an efficient embedded gesture classifier with a control strategy tailored for an intuitive interaction between the user and the prosthesis. We demonstrate that this combined approach outperforms systems based on mere pattern recognition, since they target the accuracy of a classification algorithm rather than the control of a gesture. The system was fully implemented, tested on healthy and amputee subjects and compared against benchmark repositories. The proposed approach achieves an error rate of 1.6% in the end-to-end real time control of commonly used hand gestures, while complying with the power and performance budget of a low-cost microcontroller.
BibTeX
@inproceedings{benatti2017prosthetic,
  author = {Simone Benatti and Bojan Milosevic and Elisabetta Farella and Emanuele Gruppioni and Luca Benini},
  title = {{A Prosthetic Hand Body Area Controller Based on Efficient Pattern Recognition Control Strategies}},
  booktitle = {Sensors},
  year = {2017},
  url = {https://openreview.net/forum?id=yqRXqskLbh},
}
Figure