Multiday EMG-Based Classification of Hand Motions with Deep Learning Techniques

Muhammad Zia ur Rehman, Asim Waris, Syed Omer Gilani, Mads Jochumsen, Imran Khan Niazi, Mohsin Jamil, Dario Farina, Ernest Nlandu KamavuakoSensors 2018 · DOI

TL;DR

This paper explores the use of deep learning, specifically Convolutional Neural Networks (CNNs), for classifying hand movements from electromyography (EMG) signals recorded over multiple days. The key innovation is using raw EMG signals directly as input to the CNN, demonstrating its potential to overcome the challenges of feature extraction and improve long-term robustness in myoelectric control for prosthetics.

What Problem Does This Paper Address?

Controlling prosthetic limbs with electrical signals from remaining muscles (EMG) is a long-standing goal. While conventional myoelectric control is limited, pattern recognition (PR)-based approaches offer more sophisticated control. However, a major hurdle in PR-based systems is the selection and extraction of features from EMG signals, which are inherently noisy and prone to variations over time due to factors like electrode placement and user fatigue. These variations can significantly degrade classification performance, limiting the long-term usability of such systems. Existing deep learning approaches often focus on short-term performance or require pre-processed features, failing to address the critical issue of robustness over extended periods (days). This paper aims to bridge this gap by investigating whether deep learning, particularly CNNs, can effectively learn from raw EMG data over multiple days to achieve robust and reliable hand motion classification.

Methods

The study proposes using a Convolutional Neural Network (CNN) architecture for classifying hand movements. Unlike traditional methods that rely on handcrafted features (like Mean Absolute Value, Waveform Length, Slope Sign Change, and Zero Crossing), this CNN takes raw bipolar EMG samples as direct input. This "data-driven" approach allows the network to learn relevant features intrinsically.

To benchmark the CNN's performance, two other methods were used:

  • Linear Discriminant Analysis (LDA): A classical machine learning algorithm, often used with handcrafted features.
  • Stacked Sparse Autoencoders (SSAE): Another deep learning technique, evaluated in two ways:
  • SSAE-f: Using handcrafted features as input.
  • SSAE-r: Using raw EMG samples as input, similar to the CNN.

The classification performance was evaluated using mean classification error (CE), with lower CE indicating better performance. Various cross-validation schemes were employed to assess performance within sessions, between sessions on the same day, between pairs of days, and a leave-one-out approach for all 15 days.

Datasets

The study utilized EMG data collected from seven able-bodied subjects over 15 consecutive days. For each subject, two sessions of data were recorded per day. The subjects performed seven distinct hand movements (close hand, open hand, wrist flexion, wrist extension, pronation, supination, and rest). The data was collected using a commercial MYO Armband, sampling at 200 Hz. This dataset, collected over an extended period with multiple sessions per day, serves as a benchmark for evaluating the long-term robustness of the classification algorithms.

Key Insights and Ideas

  • Raw EMG for Deep Learning is Promising: The study strongly suggests that deep learning models, particularly CNNs, can effectively learn from raw EMG signals without the need for manual feature engineering, which is a significant bottleneck in traditional myoelectric control.
  • CNNs Excel in Long-Term Robustness: The CNN demonstrated superior performance in classifying hand movements over multiple days, outperforming classical methods like LDA and even other deep learning approaches like SSAE when using raw data. This highlights the potential of CNNs to adapt to the inherent variability of EMG signals over time.
  • Data-Driven Features Outperform Handcrafted: The results indicate that the features learned by the CNN directly from raw EMG data are more effective for long-term classification than handcrafted features, even when used with advanced models like SSAE.
  • SSAE Needs Handcrafted Features: While SSAE is a deep learning technique, its performance with raw EMG (SSAE-r) was poor. It required handcrafted features (SSAE-f) to achieve competitive results, suggesting it's less adept at learning complex temporal patterns directly from raw signals compared to CNNs in this context.

Key Results

  • Within-Session Analysis: CNN and SSAE-f showed no significant difference, but both significantly outperformed LDA and SSAE-r.
  • Between-Sessions Analysis: CNN and SSAE-f again performed similarly and significantly better than LDA and SSAE-r.
  • Between Pairs of Days Analysis: CNN consistently outperformed SSAE-r, and SSAE-f outperformed LDA.
  • Leave-One-Out Between Days Analysis: CNN achieved the lowest absolute classification error (CE), with no significant difference between CNN and SSAE-f. Both significantly outperformed SSAE-r and LDA.
  • Overall Performance: CNN achieved the lowest overall mean CE (9.79 ± 4.57), followed by SSAE-f (10.98 ± 5.47), LDA (14.73 ± 6.32), and SSAE-r (25.18 ± 3.93). Statistical tests confirmed CNN outperformed all other classifiers.
  • Computational Efficiency: CNN was found to be more computationally efficient in terms of training and testing time compared to SSAE-f.

Limitations and Future Work

  • Limited Subject Pool: The study involved a small number of able-bodied subjects (seven). This limits the generalizability of the findings to a broader population, including individuals with limb loss.
  • Offline Analysis: The experiments were conducted offline. Future work should focus on real-time implementation to assess the practical usability and performance in a dynamic environment.
  • Generalizability: The authors acknowledge that the study is a preliminary step and that the findings may not be directly generalizable due to the limited subject pool and offline analysis.
  • Further Exploration of Deep Learning Architectures: While CNN showed promise, further research could explore other deep learning architectures or hybrid models to potentially improve performance even further.
  • Adaptive Learning: Although the study focused on long-term data, it did not explicitly implement adaptive learning strategies. Investigating adaptive algorithms that can continuously adjust to user changes over time could further enhance robustness.
  • Clinical Translation: The ultimate goal is clinical translation, which requires extensive validation with amputee populations and real-world prosthetic integration.

Summary updated

BibTeX
@article{rehman2018multiday,
  author = {Muhammad Zia ur Rehman and Asim Waris and Syed Omer Gilani and Mads Jochumsen and Imran Khan Niazi and Mohsin Jamil and Dario Farina and Ernest Nlandu Kamavuako},
  title = {{Multiday EMG-Based Classification of Hand Motions with Deep Learning Techniques}},
  journal = {Sensors},
  year = {2018},
  doi = {10.3390/s18082497},
}
Figure