Continuous Prediction of Lower-Limb Kinematics From Multi-Modal Biomedical Signals

Chunzhi Yi, Feng Jiang, Shengping Zhang, Hao Guo, Chifu Yang, Zhen Ding, Baichun Wei, Xiangyuan Lan, Huiyu Zhou

Abstract

The fast-growing techniques of measuring and fusing multi-modal biomedical signals enable advanced motor intent decoding schemes of lower-limb exoskeletons, meeting the increasing demand for rehabilitative or assistive applications. Challenges remain in making continuous predictions to compensate for the hysteretic response caused by mechanical transmission. This paper proposes an ahead-of-time continuous prediction of lower-limb kinematics, specifically knee angles during walking. We introduce KinPreNet, an end-to-end network consisting of a feature extractor and an angle predictor. By exploiting the electromechanical delay (EMD), we explore the capability of compensating for mechanical transmission delays. Experiments using EMG and IMU signals from ten subjects demonstrate that our approach significantly outperforms traditional methods.

The Challenge: Overcoming Mechanical Lag

Lower-limb exoskeletons are transformative for rehabilitation, yet they often suffer from a perceptible "lag." This delay stems from the mechanical transmission—the time required for motors and gears to react to a control signal. Even a few dozen milliseconds of delay can make an exoskeleton feel cumbersome or unstable for the wearer. To achieve seamless assistance, the system must predict the user's intended movement before it physically occurs. While previous methods focused on discrete gait phases, they lacked the continuity and look-ahead range necessary to fully mask mechanical latency.

The KinPreNet Framework

To address this, we developed KinPreNet, a deep learning architecture designed for continuous, ahead-of-time kinematics prediction. The system utilizes two primary components:

  1. LSTM Feature Extractor: Rather than relying on manually engineered features from Electromyography (EMG) signals, this component uses a Long Short-Term Memory (LSTM) network to learn the complex, non-linear patterns of muscle activation across nine key lower-limb muscles.
  2. LSTM Motion Predictor: This stage fuses the learned muscle features with current knee angle data from Inertial Measurement Units (IMUs) to forecast the knee angle at a future time point.
Overview of the KinPreNet architecture and the stereo photogrammetric supervision framework.

The training process employs a "Teacher-Student" paradigm. A high-precision stereo-photogrammetric system (Vicon) serves as the "Teacher," providing ground-truth joint angles. KinPreNet (the "Student") learns to predict these angles using only wearable, non-invasive sensors.

Exploiting the Electromechanical Delay (EMD)

A core innovation of this work is the strategic use of the Electromechanical Delay (EMD). In human physiology, there is a natural interval (typically 25–125ms) between the electrical activation of a muscle and the resulting mechanical force. By capturing EMG signals, KinPreNet taps into this physiological "buffer," allowing the system to "see" the intended movement before the limb actually moves. Our research identified a reliable prediction "sweet spot" between 27ms and 108ms, which aligns perfectly with the requirements for compensating robotic mechanical lag.

Experimental Validation

The model was validated using data from 10 healthy subjects. The dataset included over 90 million EMG samples from nine muscles (including the rectus femoris and gastrocnemius) and 670,000 IMU-derived knee angle points. We compared the performance of KinPreNet's learned features against traditional hand-crafted features and compared the LSTM predictor against standard Support Vector Machine (SVM) approaches.

Performance and Results

KinPreNet demonstrated exceptional accuracy, achieving an average Root Mean Square Error (RMSE) of 3.98 degrees. This is a significant improvement over traditional ahead-of-time prediction methods, which yielded an average error of approximately 15.95 degrees.

Comparison of Root Mean Square Error (RMSE) between traditional SVM and the proposed LSTM-based KinPreNet.

Furthermore, the model maintained a high Pearson correlation (R-value of 0.88) for 100ms predictions. With an inference time of just 5–7ms, the algorithm is well-suited for real-time implementation on embedded hardware like the NVIDIA TX2.

Conclusion and Future Directions

By leveraging multi-modal signal fusion and the natural physiological delay of human movement, KinPreNet provides a robust solution for the mechanical lag in assistive robotics. While currently tested on healthy subjects during level walking, future work will focus on personalizing the model for individuals with mobility impairments and extending the prediction capabilities to more complex terrains and additional joints, such as the hip and ankle.

Summary updated

Abstract

The fast-growing techniques of measuring and fusing multi-modal biomedical signals enable advanced motor intent decoding schemes of lowerlimb exoskeletons, meeting the increasing demand for rehabilitative or assistive applications of take-home healthcare. Challenges of exoskeletons motor intent decoding schemes remain in making a continuous prediction to compensate for the hysteretic response caused by mechanical transmission. In this paper, we solve this problem by proposing an ahead of time continuous prediction of lower limb kinematics, with the prediction of knee angles during level walking as a case study. Firstly, an end-to-end kinematics prediction network(KinPreNet), consisting of a feature extractor and an angle predictor, is proposed and experimentally compared with features and methods traditionally used in ahead-of-time prediction of gait phases. Secondly, inspired by the electromechanical delay(EMD), we further explore our algorithm's capability of compensating response delay of mechanical transmission by validating the performance of the different sections of prediction time. And we experimentally reveal the time boundary of compensating the hysteretic response. Thirdly, a comparison of employing EMG signals or not is performed to reveal the EMG and kinematic signals collaborated contributions to the continuous prediction. During the experiments, EMG signals of nine muscles and knee angles calculated from inertial measurement unit (IMU) signals are recorded from ten healthy subjects. To the best of our knowledge, this is the first study of continuously predicting lower-limb kinematics in an ahead-of-time manner based on the electromechanical delay (EMD).
BibTeX
@article{yi2022continuous,
  author = {Chunzhi Yi and Feng Jiang and Shengping Zhang and Hao Guo and Chifu Yang and Zhen Ding and Baichun Wei and Xiangyuan Lan and Huiyu Zhou},
  title = {{Continuous Prediction of Lower-Limb Kinematics From Multi-Modal Biomedical Signals}},
  journal = {IEEE Trans. Circuits Syst. Video Technol.},
  year = {2022},
  url = {https://openreview.net/forum?id=bcVThZZ-uKM},
}
Figure