Intelligent Human-Computer Interaction Based on Surface EMG Gesture Recognition

Jinxian Qi, Guozhang Jiang, Gongfa Li, Ying Sun, Bo TaoIEEE Access 2019 · DOI

Abstract

Urban intelligence is an emerging concept which guides a series of infrastructure developments in modern smart cities. Human-computer interaction (HCI) is the interface between residents and the smart cities, it plays a key role in bridging the gap in applicating information technologies in modern cities. Hand gestures have been widely acknowledged as a promising HCI method, recognition human hand gestures using surface electromyogram (sEMG) is an important research topic in the application of sEMG. However, state-of-the-art signal processing technologies are not robust in feature extraction and pattern recognition with sEMG signals, several technical problems are still yet to be solved. For example, how to maintain the availability of myoelectric control in intermittent use, since pattern recognition qualities are greatly affected by time variability, but it is unavoidable during daily use. How to ensure the reliability and effectiveness of myoelectric control system also important in developing a good human-machine interface. In this paper, linear discriminant analysis (LDA) and extreme learning machine (ELM) are implemented in hand gesture recognition system, which is able to reduce the redundant information in sEMG signals and improve recognition efficiency and accuracy. The characteristic map slope (CMS) is extracted by using the feature re-extraction method because CMS can strengthen the relationship of features cross time domain and enhance the feasibility of cross-time identification. This study is focusing on optimizing the time differences in sEMG pattern recognition, the experimental results are beneficial to reducing the time differences in gesture recognition based on sEMG. The recognition framework proposed in this paper can enhance the generalization ability of HCI in the long term use and it also simplifies the data collection stage before training the device ready for daily use, which is of great significance to improve the time generalization performance of an HCI system.

Introduction

Surface electromyography (sEMG), which detects the electrical activity of muscles through the skin, represents a significant opportunity for intuitive human-computer interaction (HCI). It enables hands-free control for applications ranging from prosthetic limbs to smart home interfaces. However, sEMG signals are inherently non-stationary; their statistical properties fluctuate over time due to factors like muscle fatigue, electrode displacement, and skin moisture changes. This "time gap" often necessitates frequent recalibration, as a system trained on one day may see a significant performance drop on subsequent days. This research addresses this challenge by proposing a method that generalizes well across different time periods without constant retraining.

Methods

The proposed recognition framework utilizes a multi-stage pipeline to process raw muscle signals into reliable commands:

  1. Feature Extraction and Reduction: Initial features—Root Mean Square (RMS), Waveform Length (WL), and Median Amplitude Spectrum (MAS)—are extracted. Linear Discriminant Analysis (LDA) is then applied to compress these features, eliminating redundancy while maximizing the separability between gesture classes.
  2. Characteristic Map Slope (CMS): To stabilize features against temporal drift, the researchers introduced the CMS. Instead of relying on raw magnitudes, CMS calculates the "slope" or relationship between adjacent dimensions in the fused feature map. This captures the underlying signal trends, which remain more consistent over time than absolute values.
  3. Classification via GA-ELM: An Extreme Learning Machine (ELM) is used for classification due to its high training speed. To mitigate the ELM's sensitivity to initial random weights, a Genetic Algorithm (GA) is employed to optimize the starting weights and thresholds, ensuring the model avoids local optima.

Experimental Setup

The study utilized a custom dataset of nine static gestures, including wrist movements, finger movements, and palm opening/closing.

Examples of static gestures used in the study: SH (fist), SK (open palm), and NQ (wrist flexion).
Additional gestures from the dataset: MW, MX, and RE, showing various finger and wrist configurations.

Data was collected over three days. To evaluate "time generalization," the model was trained on data from the first two days and tested on data from the third day.

Results and Discussion

The experimental results demonstrate the effectiveness of the proposed framework in bridging the time gap:

  • Baseline (Standard LDA): Achieved a cross-day accuracy of 67.18%.
  • LDA + CMS: The introduction of the Characteristic Map Slope improved accuracy to 75.74%, validating the stability of trend-based features.
  • Full Framework (LDA + CMS + GA-ELM): The complete optimized system reached a final cross-day accuracy of 79.32%.

The use of CMS significantly reduced the discretization error between training and testing sets from different days, making the system more viable for long-term, intermittent use.

Conclusion

This research presents a robust framework for sEMG-based gesture recognition that addresses the critical issue of temporal signal variability. By combining trend-based feature re-extraction (CMS) with evolutionary optimization of neural networks (GA-ELM), the system achieves nearly 80% accuracy in cross-day testing. While further improvements are needed for seamless consumer use—particularly for dynamic movements—this work provides a significant step toward reliable, long-term HCI in smart city environments.

Summary updated

Abstract

Urban intelligence is an emerging concept which guides a series of infrastructure developments in modern smart cities. Human-computer interaction (HCI) is the interface between residents and the smart cities, it plays a key role in bridging the gap in applicating information technologies in modern cities. Hand gestures have been widely acknowledged as a promising HCI method, recognition human hand gestures using surface electromyogram (sEMG) is an important research topic in the application of sEMG. However, state-of-the-art signal processing technologies are not robust in feature extraction and pattern recognition with sEMG signals, several technical problems are still yet to be solved. For example, how to maintain the availability of myoelectric control in intermittent use, since pattern recognition qualities are greatly affected by time variability, but it is unavoidable during daily use. How to ensure the reliability and effectiveness of myoelectric control system also important in developing a good human-machine interface. In this paper, linear discriminant analysis (LDA) and extreme learning machine (ELM) are implemented in hand gesture recognition system, which is able to reduce the redundant information in sEMG signals and improve recognition efficiency and accuracy. The characteristic map slope (CMS) is extracted by using the feature re-extraction method because CMS can strengthen the relationship of features cross time domain and enhance the feasibility of cross-time identification. This study is focusing on optimizing the time differences in sEMG pattern recognition, the experimental results are beneficial to reducing the time differences in gesture recognition based on sEMG. The recognition framework proposed in this paper can enhance the generalization ability of HCI in the long term use and it also simplifies the data collection stage before training the device ready for daily use, which is of great significance to improve the time generalization performance of an HCI system.
BibTeX
@inproceedings{qi2019intelligent,
  author = {Jinxian Qi and Guozhang Jiang and Gongfa Li and Ying Sun and Bo Tao},
  title = {{Intelligent Human-Computer Interaction Based on Surface EMG Gesture Recognition}},
  booktitle = {IEEE Access},
  year = {2019},
  doi = {10.1109/access.2019.2914728},
}
Figure