Lower Body Kinematics Monitoring in Running Using Fabric-Based Wearable Sensors and Deep Convolutional Neural Networks

Mohsen Gholami, Ahmad Rezaei, Tyler J. Cuthbert, Christopher Napier, Carlo MenonSensors 2019 · OpenReview

TL;DR

This paper presents a novel system using fabric-based strain sensors and deep convolutional neural networks (CNNs) to accurately track lower body kinematics (joint angles) during running. This offers a more practical and comfortable alternative to lab-based motion capture for continuous gait analysis, potentially aiding in injury prevention and performance improvement.

What Problem Does This Paper Address?

Accurate monitoring of running kinematics is vital for understanding injury risks and optimizing performance. Traditional methods like motion capture systems, while highly accurate, are confined to laboratory settings, making them impractical for real-time, everyday training. Inertial Measurement Units (IMUs) are a common wearable alternative, but they face challenges with multi-axis joint angle measurement, susceptibility to magnetic disturbances, and differences between bone and sensor coordinate systems. This paper addresses the need for an unobtrusive, flexible, and accurate wearable system capable of capturing 3D lower body kinematics during running, specifically focusing on the limitations of existing wearable technologies.

Methods

The core of this research lies in the integration of two key components: fabric-based strain sensors and a deep convolutional neural network (CNN).

  • Fabric-Based Strain Sensors: The researchers developed a prototype using fiber strain sensors made from a spandex yarn coated with a carbon black thermoplastic elastomer composite. These sensors are flexible and comfortable, offering an advantage over rigid IMUs.
  • Sensor Placement Optimization: To maximize accuracy, the placement of sensors on the pelvis was optimized using a genetic algorithm (GA) and a sequential forward (FS) method. This approach treated sensor positions as features and aimed to find the optimal subset for accurate joint angle estimation.
  • Deep Convolutional Neural Network (CNN): A four-layer CNN architecture was implemented using Keras and TensorFlow. This network was designed to process raw sensor signals, along with their first and second derivatives, to estimate joint angles in the sagittal, frontal, and transverse planes. The CNN was trained and evaluated in both "intra-participant" (model trained and tested on the same individual) and "inter-participant" (model trained on multiple individuals and tested on a new one) scenarios.

Datasets

The study utilized data collected from ten healthy male participants running on a split-belt instrumented treadmill at various speeds (8-12 km/h). For each participant, 15 minutes of running data at 100 Hz was collected, resulting in approximately 90,000 data samples from the nine sensors. This data was used for both training and testing the CNN models. The motion capture system served as the ground truth for evaluating the accuracy of the strain sensor system.

Key Insights and Ideas

  • Sensor Optimization is Crucial: The study highlights that the precise placement of sensors is critical for achieving high accuracy. The genetic algorithm proved more effective than the sequential forward method in finding optimal sensor configurations.
  • CNNs Excel at Complex Patterns: Deep CNNs are shown to be highly effective in extracting features from complex sensor signals and predicting nonlinear patterns (like hysteresis) inherent in piezoresistive sensors, leading to improved accuracy, especially at higher running speeds.
  • 3D Kinematics are Achievable: The system successfully estimates joint angles in all three planes (sagittal, frontal, transverse), expanding beyond the typical sagittal-plane focus of many previous wearable sensor studies.
  • Intra-Participant vs. Inter-Participant Performance: While intra-participant models achieved higher accuracy (lower RMSE), inter-participant models demonstrated the potential for a "one-size-fits-all" solution, eliminating the need for individual calibration, which is a significant practical advantage.
  • Sagittal Plane Dominance: Accuracy was generally higher for sagittal plane joint angles compared to frontal and transverse planes. This is attributed to the larger range of motion in the sagittal plane, making it more sensitive to sensor strain, and potentially the inherent complexity of non-sagittal movements.

Key Results

  • Intra-Participant Accuracy: RMSE values were less than 2.2° and NRMSE values were less than 5.3% for hip, knee, and ankle joints across all planes.
  • Inter-Participant Accuracy: RMSE values were less than 6.4° and NRMSE values were less than 10% for sagittal plane joint angles.
  • Comparison to Previous Work: The achieved RMSE values for knee, hip, and ankle joints (1.12°, 2.2°, and 1.3° respectively) significantly outperformed previous studies using hyper-elastic sensors and capacitive strain sensors.
  • Speed Consistency: The system demonstrated consistent performance across different running speeds, a notable improvement over some prior research that showed increased errors at higher speeds.

Limitations and Future Work

  • Cohort Diversity: The study cohort consisted of only ten healthy male participants. Future work should expand to include a more diverse group (different sexes, ages, body types) to test the CNN model's generalization capabilities with varying training datasets.
  • Testing Environment: The experiments were conducted in a laboratory setting on a treadmill. Outdoor running, with its varied terrain and gait patterns, could present different challenges.
  • Short Testing Periods: The data collection periods were limited. Longer-term testing would be beneficial to assess garment fit and performance over extended use.
  • Washability and Durability: While the sensors are fabric-based, the paper doesn't explicitly detail the washability and long-term durability of the system, which are crucial for practical applications.
  • Further Applications: The authors suggest exploring the system's potential for other applications like gait phase detection, rehabilitation monitoring, and Parkinson's disease monitoring.
  • Wireless Capabilities: The current setup requires wired connections for data acquisition. Developing wireless capabilities would further enhance the system's practicality.

Summary updated

Abstract

Continuous kinematic monitoring of runners is crucial to inform runners of inappropriate running habits. Motion capture systems are the gold standard for gait analysis, but they are spatially limited to laboratories. Recently, wearable sensors have gained attention as an unobtrusive method to analyze performance metrics and the health conditions of runners. In this study, we developed a system capable of estimating joint angles in sagittal, frontal, and transverse planes during running. A prototype with fiber strain sensors was fabricated. The positions of the sensors on the pelvis were optimized using a genetic algorithm. A cohort of ten people completed 15 min of running at five different speeds for gait analysis by our prototype device. The joint angles were estimated by a deep convolutional neural network in inter- and intra-participant scenarios. In intra-participant tests, root mean square error (RMSE) and normalized root mean square error (NRMSE) of less than 2.2° and 5.3%, respectively, were obtained for hip, knee, and ankle joints in sagittal, frontal, and transverse planes. The RMSE and NRMSE in inter-participant tests were less than 6.4° and 10%, respectively, in the sagittal plane. The accuracy of this device and methodology could yield potential applications as a soft wearable device for gait monitoring of runners.
BibTeX
@inproceedings{gholami2019lower,
  author = {Mohsen Gholami and Ahmad Rezaei and Tyler J. Cuthbert and Christopher Napier and Carlo Menon},
  title = {{Lower Body Kinematics Monitoring in Running Using Fabric-Based Wearable Sensors and Deep Convolutional Neural Networks}},
  booktitle = {Sensors},
  year = {2019},
  url = {https://openreview.net/forum?id=yAd1-uErMO},
}
Figure