Abstract
Reliable hand gesture recognition (HGR) is a cornerstone for advanced Human-Machine Interfaces (HMIs) in prosthetics. While traditional methods rely on macroscopic surface Electromyogram (sEMG) signals, recent advances allow for microscopic neural drive analysis. This paper introduces HYDRA-HGR, a hybrid framework that fuses these two information levels using a dual-path Vision Transformer (ViT) architecture. Our results demonstrate that this fusion achieves a high accuracy of 94.86%, significantly outperforming models restricted to a single data type.
Introduction
The development of robust HMIs for "Cyber-Physical-Human" systems depends on the accurate interpretation of muscle activity. Historically, research has focused on the macroscopic level—the raw electrical activity recorded from the skin. However, High-Density sEMG (HD-sEMG) now enables microscopic analysis, which involves decomposing signals into individual motor unit activities. Despite its potential, this neural drive information is often overlooked due to computational complexity. HYDRA-HGR addresses this by integrating both the "broad strokes" of muscle activity and the "fine details" of neural instructions into a unified recognition model.
Methodology
The HYDRA-HGR architecture consists of two parallel Vision Transformer paths designed to capture complementary features:
- The Macro Path: Processes raw HD-sEMG signals as 3D spatial-temporal images (time × 2D electrode grid). A ViT extracts broad spatial and temporal features from these muscle activation patterns.
- The Micro Path: Utilizes Blind Source Separation (BSS) to decompose HD-sEMG signals into individual Motor Unit Action Potentials (MUAPs). These are transformed into 2D "peak-to-peak" images representing the activation levels of specific neural sources, which are then processed by a second ViT.
The final classification is performed by concatenating the "class tokens" from both paths and passing them through a fully connected (FC) layer. This synergy allows the model to leverage the unique signatures present in both macroscopic and microscopic data.
Experimental Setup
The framework was evaluated using a large-scale HD-sEMG dataset involving 20 non-disabled participants performing 65 isometric hand movements (66 total gestures including repetitions). Data was recorded using two 8×8 electrode grids (128 channels) sampled at 2,048 Hz. The model's performance was validated using a 5-fold cross-validation scheme.
Results and Discussion
HYDRA-HGR achieved an average accuracy of 94.86% within a 250 ms window. This performance represents a significant improvement over standalone models: a 5.52% gain over the Macro-only path and an 8.22% gain over the Micro-only path.
The results indicate that macroscopic and microscopic information are complementary rather than redundant. The Macro path captures muscle geometry and intensity, while the Micro path identifies specific neural signatures. Notably, the hybrid model also demonstrated lower variance and higher stability across different subjects, as illustrated in the comparative boxplots.
Conclusion and Future Work
While HYDRA-HGR sets a new benchmark for HGR accuracy, challenges remain regarding the computational overhead of the decomposition stage (BSS) for real-time applications. Future research will focus on optimizing inference latency and validating the architecture on diverse populations, including amputees, to ensure the system's robustness in clinical settings.
Summary updated