TL;DR
This paper introduces a novel hybrid deep learning approach for estimating human velocity from a single Inertial Measurement Unit (IMU). By combining a ResNet for spatial feature extraction with a Bidirectional LSTM (BLSTM) for temporal tracking, and further refining the estimates with a Linear Kalman Filter (LKF), the method achieves state-of-the-art accuracy and robustness, even in highly dynamic scenarios and with non-rigid sensor placement.
What Problem Does This Paper Address?
Pedestrian Dead Reckoning (PDR) is a crucial technique for estimating a person's position using IMUs. However, accurately estimating velocity from IMU data is notoriously challenging due to sensor noise and drift, which accumulate over time. Traditional PDR methods often require specific sensor placement (e.g., on the feet or spine) and extensive parameter tuning, making them unsuitable for many real-world applications where sensors are placed casually (e.g., in a pocket). Existing data-driven methods, while promising, often struggle with highly dynamic movements and varying motion states. This paper aims to bridge this gap by developing a robust and generalizable velocity estimation method that works with a single, uncalibrated IMU, even during complex and dynamic human activities.
Methods
The authors propose a three-step processing pipeline:
- Data Pre-processing: Raw IMU data (accelerometer and gyroscope) is synchronized, cleaned, resampled, and interpolated. The core of this step involves creating "window bundles" from the data, where each window represents a short segment of motion.
- Main Processing (Hybrid DL Architecture):
- Spatial Feature Extraction: A modified ResNet-18 architecture is used to extract high-dimensional spatial features ($f_s$) from each input window.
- Temporal Feature Tracking: These spatial features are then fed into a "Temporal Feature Tracker" composed of two Bidirectional Long Short-Term Memory (BLSTM) layers. The BLSTMs capture the temporal dependencies ($f_t$) between features, allowing the model to understand the dynamics of motion over time.
- The output of the BLSTM is a predicted velocity ($v'$).
- Post-Processing (Linear Kalman Filter - LKF): The predicted velocities ($v'$) from the DL model are further optimized by a Linear Kalman Filter (LKF). The LKF helps to reduce noise and improve the accuracy and precision of the velocity estimates.
The key novelty lies in the combination of a deep convolutional network (ResNet) for robust spatial feature extraction, a BLSTM for capturing complex temporal dynamics, and an LKF for final refinement, all operating on rotation-invariant signals (magnitude of acceleration and angular rate).
Datasets
The authors collected a novel dataset for their experiments.
- Dataset Acquisition: Data was collected from 23 participants performing four types of activities: walking, jogging, running, and random movements (a combination of the first three). Participants wore two smartphones (one in each pocket) to collect data. The ground truth was captured using a millimeter-accurate optical Qualisys motion tracking system at 100 Hz.
- Dataset Variants: Three variants were created:
- V1: Covers the entire data for each subject and activity, with varying durations.
- V2: Trims activities to have the same duration, reducing redundancy.
- V3: Homogenizes velocity groups across all users and activities, ensuring a more balanced distribution of velocities for training.
- Training/Validation/Testing Split: Data was split into 70% training, 10% validation, and 20% testing sets. Specific splits were used for accuracy, computational effort, and generalizability benchmarks.
Key Insights and Ideas
- The Power of Hybrid Models: Combining spatial feature extraction (ResNet) with temporal modeling (BLSTM) is crucial for accurately estimating velocity, especially in dynamic scenarios. The temporal component ($f_t$) significantly outperforms methods that only rely on spatial features ($f_s$).
- Rotation Invariance is Key: By using the Signal Magnitude Vector (SMV) of the IMU data, the proposed method achieves rotation invariance, meaning it doesn't require specific sensor orientation or calibration, making it more practical for real-world use.
- LKF as a Powerful Refiner: Adding a Linear Kalman Filter as a post-processing step significantly improves the accuracy and robustness of velocity estimates for all tested methods, particularly for those relying solely on spatial features.
- Dataset Structure Matters: The performance of data-driven methods is highly sensitive to dataset characteristics. A dataset with homogeneous velocity distributions (like V3) leads to better generalization and accuracy compared to datasets with highly variable durations or velocities.
- Extrapolation is Difficult: Data-driven methods struggle to extrapolate to velocities or motion patterns not present in the training data.
Key Results
- State-of-the-Art Accuracy: The proposed Hybrid method achieved the best performance across various metrics. On the test set, it achieved a Mean Absolute Error (MAE) of 0.25 m/s and a distance error of 9.46 m per kilometer (DET).
- Robustness: The Hybrid method demonstrated strong robustness against different movement states and orientation changes.
- Generalizability: The method generalized well to unknown users and sensor types, performing comparably to or better than other methods on left-out users and devices.
- Computational Efficiency: While not the fastest in terms of live inference time (PDR was fastest), the Hybrid method offered a good balance between accuracy and computational cost, especially when considering its superior performance.
- LKF Impact: The LKF improved accuracy by an average of 145% on V1, 70% on V2, and 60% on V3.
Limitations and Future Work
- Extrapolation: The paper highlights that none of the tested methods, including the proposed Hybrid, can effectively extrapolate to velocities or motion patterns significantly outside the training data distribution.
- Sensor Placement: While the method is robust to non-rigid placement, it still relies on IMU data. Future work could explore combining IMU data with other sensor modalities (e.g., low-frequency radio-based positions) for even more robust localization.
- Other Movements: The authors plan to investigate other types of movements, such as jumping and climbing stairs, and examine natural IMU positions like head and hands.
- Synthetic Data: Future work will explore training on synthetic data and retraining on real data to further push the boundaries of generality.
- Multi-Sensor Fusion: Combining the Hybrid method with PDR for multi-sensor fusion and comparing it to state-of-the-art PDR methods is also a future direction.
Summary updated