sEMG-Based Gesture Recognition with Convolution Neural Networks

Zhen Ding, Chifu Yang, Zhihong Tian, Chunzhi Yi, Yunsheng Fu, Feng JiangSustainability 2018 · DOI

TL;DR

This paper introduces a novel Convolutional Neural Network (CNN) architecture for surface electromyography (sEMG)-based gesture recognition. By employing parallel convolutional blocks with larger kernel sizes and explicitly considering muscle independence, their proposed method achieves state-of-the-art accuracy on a benchmark dataset, outperforming traditional methods and existing CNN approaches.

What Problem Does This Paper Address?

Recognizing human limb movements from surface electromyography (sEMG) signals is crucial for applications like advanced prosthetics and human-computer interfaces. Traditional methods often rely on hand-crafted feature extraction, which can lead to information loss and reduced accuracy. While deep learning, specifically CNNs, has shown promise by automatically learning features, existing CNN architectures for sEMG often use kernel sizes optimized for image processing, which may not be ideal for the unique characteristics of sEMG signals. Furthermore, these architectures sometimes overlook the inherent independence of different muscles, potentially impacting recognition performance. This paper aims to bridge this gap by proposing a CNN architecture that better leverages sEMG signal properties for improved gesture recognition accuracy.

Methods

The core of this work is a novel CNN architecture called "Convolution with two Parallel Block" (C-B1PB2). This architecture features two parallel "blocks" that process the sEMG data independently before their outputs are concatenated and fed into a classifier.

  • Parallel Multiple-Scale Convolution: Block 1 uses smaller kernel sizes (1x13, 7x13, 6x13), while Block 2 employs larger kernel sizes (1x57, 7x13, 6x13). This parallel structure allows the network to capture features at different scales.
  • Larger Kernel Sizes: Unlike many existing CNNs for sEMG that use small kernels (e.g., 3x3), this paper argues for and utilizes larger kernel sizes, particularly in the second dimension (sampling points), to better capture the temporal dynamics of sEMG signals.
  • Muscle Independence Consideration: The architecture processes each sEMG channel independently in the initial convolutional layers of the parallel blocks. This approach explicitly accounts for the statistical independence of signals from different muscles, preventing premature fusion of information that could lead to errors. Information from different electrodes is only mixed in later layers.
  • Feature Extractor and Classifier: The architecture consists of a feature extractor (the two parallel blocks) and a classifier (three fully connected layers with a softmax output). Batch normalization is used within the blocks, and dropout is applied in the classifier.

Datasets

The primary dataset used for training and evaluation is the NinaPro database, specifically DB2. This database contains sEMG data from 40 subjects performing 50 types of hand, wrist, and functional movements. The authors focused on 17 hand and wrist movements from Exercise B for their experiments. Approximately two-thirds of the repetitions were used for training, and the remaining one-third for testing.

Key Insights and Ideas

  • Kernel Size Matters for sEMG: The paper strongly suggests that kernel sizes optimized for image processing are not necessarily optimal for sEMG. Larger kernel sizes, particularly in the temporal dimension, can lead to better recognition accuracy by capturing more of the signal's dynamics.
  • Muscle Independence is Crucial: Explicitly processing individual muscle signals independently before fusion is a key architectural choice that significantly improves accuracy. This acknowledges the biological reality that different muscles contribute to movements in distinct ways.
  • Parallel Processing for Scale: Using parallel convolutional blocks with different kernel sizes allows the network to learn features at multiple temporal scales simultaneously, leading to a more robust representation.
  • End-to-End Learning Advantage: The CNN approach eliminates the need for manual feature engineering, reducing the risk of information loss inherent in traditional methods.

Key Results

The proposed C-B1PB2 architecture achieved the highest classification accuracy among all tested methods on the NinaPro DB2 Exercise B dataset, reaching 83.79%. This is a significant improvement over:

  • Classical Classification (CC): 77.96%
  • Other CNN variants: C-2B1 (82.16%), C-2B2 (82.49%), C-DK (79.23%), C-SK (75.82%), C-SK2 (81.30%).

When evaluated on all hand gestures of DB2 (Exercises B, C, and D), C-B1PB2 achieved 78.86% accuracy, which is comparable to or better than state-of-the-art methods reported in the literature, even without extensive preprocessing or long window lengths. The confusion matrix analysis revealed that certain movements (e.g., thumb up, finger flexion) were more accurately classified than others, with wrist movements showing higher accuracy for extension and supination.

Limitations and Future Work

  • Computational Complexity: While achieving high accuracy, the paper acknowledges that future work will focus on reducing the computational complexity of the proposed model.
  • Real-time Control: The authors also aim to optimize the system for real-time control applications, which is a critical aspect for practical use in prosthetics.
  • Generalizability: While tested on NinaPro DB2, further validation on other datasets and diverse subject populations would be beneficial.
  • Preprocessing Impact: The paper notes that their method achieves good results without extensive preprocessing (like low-pass filtering or FFT), which can introduce latency. However, the impact of preprocessing on real-time performance warrants further investigation.

Summary updated

Abstract

The traditional classification methods for limb motion recognition based on sEMG have been deeply researched and shown promising results. However, information loss during feature extraction reduces the recognition accuracy. To obtain higher accuracy, the deep learning method was introduced. In this paper, we propose a parallel multiple-scale convolution architecture. Compared with the state-of-art methods, the proposed architecture fully considers the characteristics of the sEMG signal. Larger sizes of kernel filter than commonly used in other CNN-based hand recognition methods are adopted. Meanwhile, the characteristics of the sEMG signal, that is, muscle independence, is considered when designing the architecture. All the classification methods were evaluated on the NinaPro database. The results show that the proposed architecture has the highest recognition accuracy. Furthermore, the results indicate that parallel multiple-scale convolution architecture with larger size of kernel filter and considering muscle independence can significantly increase the classification accuracy.
BibTeX
@inproceedings{ding2018semg,
  author = {Zhen Ding and Chifu Yang and Zhihong Tian and Chunzhi Yi and Yunsheng Fu and Feng Jiang},
  title = {{sEMG-Based Gesture Recognition with Convolution Neural Networks}},
  booktitle = {Sustainability},
  year = {2018},
  doi = {10.3390/su10061865},
}
Figure