TL;DR
This paper introduces a new benchmark dataset for high-density surface electromyography (HD-sEMG) based hand gesture recognition and proposes a deep learning framework with domain adaptation to significantly improve inter-session gesture recognition accuracy. This work addresses the critical challenge of making muscle-computer interfaces more robust and practical by reducing the need for recalibration between different usage sessions.
What Problem Does This Paper Address?
Muscle-computer interfaces (MCIs) aim to translate a user's muscle electrical activity (sEMG) into commands for external devices. A key challenge in making these interfaces practical is achieving reliable gesture recognition across different recording sessions, even for the same user. This "inter-session variability" arises from factors like slight shifts in electrode placement, changes in arm posture, and even fatigue. Existing methods often struggle with this, requiring extensive recalibration or performing poorly. Furthermore, the field has lacked a standardized, large-scale benchmark dataset for high-density sEMG (HD-sEMG) to facilitate reproducible research and development. This paper tackles both issues: creating a valuable new dataset and developing a robust deep learning approach to overcome inter-session variability.
Methods
The core of the proposed solution lies in a deep learning framework combined with domain adaptation.
- HD-sEMG to Image Conversion: Instantaneous HD-sEMG signals, captured by a 2D electrode array, are transformed into grayscale images. This allows the problem of sEMG gesture recognition to be framed as an image classification task, leveraging the power of Convolutional Neural Networks (CNNs).
- Deep Convolutional Network (ConvNet): A custom ConvNet architecture is employed, featuring convolutional layers, locally connected layers, and fully connected layers, designed to effectively process the spatial and temporal information present in the sEMG images.
- Multi-Stream Adaptive Batch Normalization (AdaBN): To address the inter-session variability, the authors adapt a technique called Adaptive Batch Normalization (AdaBN). Their novel "multi-stream" extension of AdaBN is designed to handle multiple source domains (different recording sessions) more effectively. This method incrementally updates the statistics of the BatchNorm layers to align the distributions of training and testing data without requiring labeled data from the target session. This is crucial for unsupervised adaptation in real-world scenarios.
- Unsupervised Domain Adaptation: The proposed framework primarily relies on unsupervised domain adaptation, meaning it can adapt to new sessions without requiring the user to perform specific gestures for calibration. This is a significant advantage for user experience.
Datasets
The paper introduces a new benchmark dataset and utilizes existing ones for evaluation:
- CapgMyo Database (New Benchmark): This is the primary contribution in terms of data. It comprises HD-sEMG data from 23 participants, collected using a custom-designed 8x16 electrode array. The database includes various hand gestures and is structured into three sub-databases (DB-a, DB-b, DB-c) to facilitate different types of evaluation (intra-session, inter-session, and inter-subject).
- CSL-HDEMG Database: Used for evaluating the proposed method on a different HD-sEMG dataset.
- NinaPro Database (Sub-database 1): Used for evaluating the method on a dataset with sparse multi-channel sEMG, allowing for comparison with methods designed for less dense electrode configurations.
Key Insights and Ideas
- HD-sEMG as Images: Framing HD-sEMG data as images is a powerful abstraction that unlocks the potential of deep convolutional networks for this domain, capturing both spatial and temporal patterns effectively.
- Unsupervised Domain Adaptation is Key: The success of the proposed method hinges on its ability to adapt to new sessions without explicit user calibration. This unsupervised approach is a major step towards practical, user-friendly MCIs.
- Multi-Stream AdaBN for Robustness: The multi-stream extension of AdaBN proves more stable and effective than classical AdaBN when dealing with multiple recording sessions, which is a common scenario in inter-session recognition. This suggests that handling variations across multiple sources is critical.
- New Benchmark Facilitates Progress: The creation of the CapgMyo database addresses a critical need for standardized, high-density sEMG data, enabling more reproducible and comparable research in the field.
Key Results
The proposed approach demonstrates significant improvements in inter-session gesture recognition:
- Improved Accuracy: The method achieved state-of-the-art performance on inter-session gesture recognition across multiple datasets (CSL-HDEMG, CapgMyo, and NinaPro).
- On CSL-HDEMG, the accuracy reached 82.3% with adaptation, a substantial improvement over previous methods.
- On CapgMyo DB-b and DB-c, accuracies of 35.1% and 35.1% were achieved, respectively, with adaptation.
- On NinaPro, the accuracy reached 67.4% with adaptation, a significant leap from prior results.
- Reduced Calibration Need: The unsupervised domain adaptation significantly reduces the need for explicit calibration, making the system more user-friendly.
- Effective with Limited Data: The adaptation process can be effective even with a small amount of unlabeled calibration data.
- Technical Validation: The CapgMyo database was validated, showing that state-of-the-art classifiers can achieve higher accuracies with HD-sEMG compared to sparse sEMG.
Limitations and Future Work
- Limitations:
- The paper notes that while the method works with unlabeled calibration data, fine-tuning with labeled calibration data can further improve performance.
- The current framework primarily focuses on static gestures.
- Future Work:
- The authors plan to extend their framework to incorporate temporal models, such as recurrent neural networks, to handle dynamic transitional motions in addition to static gestures.
- They also aim to accommodate more complex dynamic transitional motions.
Summary updated