The Role of Surface Electromyography in Data Fusion with Inertial Sensors to Enhance Locomotion Recognition and Prediction

Lin Meng, Jun Pang, Ziyao Wang, Rui Xu, Dong MingSensors 2021 · DOI

TL;DR

This paper demonstrates that fusing surface electromyography (sEMG) with inertial sensor data significantly improves the accuracy of recognizing human locomotion activities and, crucially, enables the prediction of upcoming locomotion modes. By leveraging the predictive power of sEMG, which activates before muscle movement, the system can anticipate gait transitions, offering a substantial advancement for human-machine interaction in areas like prosthetics and exoskeletons.

What Problem Does This Paper Address?

Accurate and timely recognition of human locomotion is vital for developing advanced assistive technologies like lower-limb exoskeletons and prosthetic limbs. These systems need to understand not just the current movement but also anticipate the user's next intended action to provide seamless and intuitive control. While inertial measurement units (IMUs) are good at capturing current motion, they struggle with predicting future movements. Surface electromyography (sEMG), which measures muscle electrical activity, has the potential to predict movement because muscle activation precedes actual limb movement. However, the specific benefit of sEMG for predicting locomotion, especially when fused with inertial data, has not been thoroughly explored. This paper aims to bridge this gap by investigating how sEMG, when combined with inertial data, can enhance both the recognition of current locomotion states and the prediction of future ones.

Methods

The researchers employed a data fusion approach, combining sEMG signals from 14 lower limb muscles with linear acceleration data from the pelvis, thigh, shank, and foot. They explored four popular machine learning classifiers: Support Vector Machine (SVM), K-Nearest Neighbor (KNN), Artificial Neural Network (ANN), and Linear Discriminant Analysis (LDA). To improve online performance, they incorporated a "major vote strategy" and a "content constraint rule" to stabilize classification decisions. The study systematically analyzed the impact of sliding window sizes on both recognition accuracy and prediction response time. They also compared the performance of using sEMG alone, acceleration alone, and the fused data.

Datasets

The study collected data from ten healthy participants performing seven different locomotion activities: sitting (S), standing (ST), level walk (LW), stair ascent (SA), stair descent (SD), ramp ascent (RA), and ramp descent (RD). The data was collected synchronously using wearable motion analysis systems (Ultium EMG and Research Pro IMU) at a sampling rate of 2000 Hz. A Vicon motion capture system served as a reference for identifying locomotion modes and gait cycles. The collected data was used for both offline training and online testing of the classification models. No new benchmark datasets were introduced; existing locomotion activities were used.

Key Insights and Ideas

  • sEMG as a Predictor: The core insight is that sEMG's ability to capture pre-movement muscle activation makes it a powerful predictor of future locomotion, even when trained solely on steady-state data. This is a significant departure from relying solely on kinematic data, which only reflects movement after it has begun.
  • Fusion is Key: Combining sEMG with inertial data offers a synergistic benefit, significantly outperforming either sensor modality alone for both recognition and prediction.
  • SVM's Dominance: Among the tested classifiers, SVM consistently demonstrated superior performance in both locomotion recognition and prediction, especially with optimized parameters.
  • Window Size Matters: The choice of sliding window size is critical. While larger windows improve accuracy, they increase response time. The study found an optimal window size (e.g., 80 ms for SVM) that balances these trade-offs effectively.
  • Predicting Transitions: The fused model could accurately predict the next steady-state locomotion mode during gait transitions, even anticipating movements like transitioning from level walking to stairs or ramps.

Key Results

  • Recognition Accuracy: Data fusion improved steady-state locomotion recognition accuracy from 90% (acceleration only) to 98% (fusion). SVM achieved the highest accuracy at 98.78 ± 1.04%.
  • Prediction Accuracy: The fused model enabled prediction of the next steady locomotion mode approximately 370 ms in advance. SVM achieved a predictive accuracy of 97.69 ± 0.85% with a response time of -372.63 ms, indicating prediction ahead of the actual event.
  • sEMG vs. Acceleration: sEMG alone provided significantly better prediction performance than linear acceleration data alone.
  • Classifier Performance: SVM outperformed KNN, LDA, and ANN in both recognition and prediction tasks. ANN did not show significant advantages over traditional machine learning methods for this specific fused-data problem.
  • Window Size Impact: Classification accuracy and predictive accuracy generally increased with window size up to a certain threshold, while response time decreased.

Limitations and Future Work

  • Feature Set: The study primarily used Root Mean Square (RMS) features for sEMG and acceleration. The authors acknowledge that other features (e.g., Wilson amplitude, energy of wavelet coefficients) could further improve performance.
  • Gait Transition Data: Data from actual gait transitions was not used for model training. Incorporating this data could potentially enhance the recognition accuracy of transitions themselves, which was lower than steady-state recognition.
  • Temporal Dynamics: The employed classifiers (SVM, KNN, LDA) may not fully capture temporal dynamics. Future work could explore recurrent neural networks or other sequence-aware models to potentially improve performance and reduce time delays.
  • Limited Classifiers: While four common classifiers were used, exploring a wider range of advanced machine learning techniques could be beneficial.
  • Participant Diversity: The study involved only ten healthy young participants. Future research should investigate performance on diverse populations, including individuals with movement disorders.

The authors suggest exploring more advanced feature fusion techniques and incorporating gait transition data into the training process to further enhance locomotion prediction and recognition capabilities.

Summary updated

Abstract

Locomotion recognition and prediction is essential for real-time human-machine interactive control. The integration of electromyography (EMG) with mechanical sensors could improve the performance of locomotion recognition. However, the potential of EMG in motion prediction is rarely discussed. This paper firstly investigated the effect of surface EMG on the prediction of locomotion while integrated with inertial data. We collected EMG signals of lower limb muscle groups and linear acceleration data of lower limb segments from ten healthy participants in seven locomotion activities. Classification models were built based on four machine learning methods-support vector machine (SVM), k-nearest neighbor (KNN), artificial neural network (ANN), and linear discriminant analysis (LDA)-where a major vote strategy and a content constraint rule were utilized for improving the online performance of the classification decision. We compared four classifiers and further investigated the effect of data fusion on the online locomotion classification. The results showed that the SVM model with a sliding window size of 80 ms achieved the best recognition performance. The fusion of EMG signals does not only improve the recognition accuracy of steady-state locomotion activity from 90% (using acceleration data only) to 98% (using data fusion) but also enables the prediction of the next steady locomotion (∼370 ms). The study demonstrates that the employment of EMG in locomotion recognition could enhance online prediction performance.
BibTeX
@inproceedings{meng2021role,
  author = {Lin Meng and Jun Pang and Ziyao Wang and Rui Xu and Dong Ming},
  title = {{The Role of Surface Electromyography in Data Fusion with Inertial Sensors to Enhance Locomotion Recognition and Prediction}},
  booktitle = {Sensors},
  year = {2021},
  doi = {10.3390/s21186291},
}
Figure