Deep Learning Methods for Speed Estimation of Bipedal Motion from Wearable IMU Sensors

Josef Justa, Václav Smídl, Ales Hamácek2022 · DOI

TL;DR

This paper explores deep learning techniques for estimating human walking and running speed using wearable Inertial Measurement Units (IMUs). They introduce a novel semi-supervised variational autoencoder (SVAE) with a unique sinusoidal decoder that outperforms existing methods and demonstrates that sensor placement and the number of sensors are more critical than the specific deep learning architecture.

What Problem Does This Paper Address?

Accurately estimating human locomotion speed is crucial for applications like indoor navigation, especially in environments where GPS or Wi-Fi signals are unreliable, such as emergency rescue scenarios. Traditional methods, often relying on manually designed features like zero-velocity updates (ZUPT), struggle with generalization, particularly at higher speeds or during complex movements like running or climbing. These methods also require significant feature engineering and can be sensitive to sensor placement. The paper aims to leverage deep learning to automatically learn these speed estimation features from raw IMU data, while also investigating the impact of sensor placement and quantity.

Methods

The researchers compare several deep learning architectures for speed estimation:

  • Feed-Forward Networks: They tested established architectures like InceptionTime and Perceiver, which are designed for time-series data. These models directly map input IMU data to speed predictions.
  • Semi-Supervised Variational Autoencoders (SVAE): This approach combines an autoencoder for regularization with a predictor. The autoencoder learns a compressed representation (latent code) of the IMU data, which is then used by a predictor to estimate speed. This helps improve generalization and prevent overfitting.
  • Conventional Decoder (SVAE-LSTM-CNN): This uses a standard LSTM-CNN architecture for reconstruction.
  • Proposed Decoder (SVAE-Sine): This is the paper's novel contribution. It uses a decoder based on weighted sine waves, motivated by the periodic nature of IMU signals. This simpler decoder was found to be more reliable.

They also evaluated conventional feature-based methods as a baseline, specifically focusing on those that don't rely on wrist sensors.

Datasets

The study introduced a new dataset collected from eight human subjects performing walking and running at various speeds. IMU data was captured from three locations on a single leg: the shoe, shin, and thigh. A custom-modified monowheel was used to provide a ground truth speed reference, offering a more natural measurement than a treadmill. This dataset, along with the code, is made publicly available.

Key Insights and Ideas

  • Sensor Placement Matters Most: The study found that the location of the IMU sensor significantly impacts accuracy. The shoe consistently provided the best results, even as a single sensor.
  • More Sensors, Better Accuracy: Adding more sensors (shin, thigh) in combination with the shoe sensor led to a monotonic decrease in estimation error. This highlights the benefit of sensor fusion for deep learning models.
  • Novel Decoder Outperforms: The proposed SVAE with a sinusoidal decoder achieved the best performance, suggesting that tailoring the architecture to the specific characteristics of IMU data can yield superior results compared to general-purpose deep learning models.
  • Deep Learning Surpasses Feature-Based Methods: All tested deep learning architectures significantly outperformed the best conventional feature-based methods, demonstrating the power of learned representations.
  • Inter-Subject Variability is a Major Challenge: The study revealed that differences in speed estimation error between individual subjects were larger than the differences between various deep learning architectures. This suggests that personalization of models might be a key area for future research.

Key Results

  • The proposed SVAE-Sine model achieved the lowest mean absolute error (MAE) in speed estimation.
  • Using a single sensor on the foot resulted in an MAE of approximately 0.4 km/h.
  • Using all three sensors (foot, shin, thigh) reduced the MAE to approximately 0.25 km/h.
  • The best conventional feature-based method achieved an MAE of around 1.2 km/h, significantly higher than the deep learning approaches.
  • Even feed-forward networks like InceptionTime provided comparable results to the SVAE models, though the SVAE-Sine was slightly superior.

Limitations and Future Work

  • Inter-Subject Variability: The authors acknowledge that individual differences in subjects are a significant source of error and suggest that personalization of deep learning models could be a promising avenue.
  • Limited Biometric Data: While they briefly explored the inclusion of leg length and body mass, these did not yield significant improvements. Further investigation into other biometric data might be beneficial.
  • Complex Motions: The paper suggests that multiple sensors will be crucial for addressing more complex human motions like knee-walking or ladder climbing, which were not extensively studied.
  • Sensor Fusion Potential: The ease of sensor fusion in deep learning models opens up possibilities for integrating data from various sensors for more robust and comprehensive motion analysis.
  • Pre-training for Personalization: The authors foresee potential in using deep learning pre-training techniques to create universal representations that can then be fine-tuned for individual users with less data and computational cost.

Summary updated

BibTeX
@misc{justa2022deep,
  author = {Josef Justa and Václav Smídl and Ales Hamácek},
  title = {{Deep Learning Methods for Speed Estimation of Bipedal Motion from Wearable IMU Sensors}},
  year = {2022},
  doi = {10.3390/s22103865},
}
Figure