TL;DR
This paper introduces "Attformer," a novel deep learning architecture leveraging self-attention mechanisms for accurate and generalizable attitude estimation in smartphones. By treating attitude estimation as an optimization problem and learning directly from raw IMU data, Attformer surpasses traditional methods and existing deep learning approaches, especially when combined with prior knowledge from Kalman filters.
What Problem Does This Paper Address?
Estimating the orientation (attitude) of a device in 3D space is crucial for numerous applications, from augmented reality to navigation. Smartphones are equipped with Inertial Measurement Units (IMUs) – accelerometers, gyroscopes, and magnetometers – that provide the raw data for this estimation. However, these sensors are notoriously noisy and prone to drift, especially the gyroscopes. Traditional methods like Extended Kalman Filters (EKF) and Unscented Kalman Filters (UKF) attempt to fuse this data but often struggle with highly dynamic movements, require extensive parameter tuning, and lack generalizability across different users, devices, and activities. Existing deep learning approaches, primarily based on Recurrent Neural Networks (RNNs), can capture short-term dependencies but fail to effectively model long-term patterns in the data, leading to accumulated errors. This paper aims to bridge this gap by developing a more robust, generalizable, and end-to-end attitude estimation solution that leverages the power of self-attention.
Methods
The core of this paper is the Attformer, a deep learning model based on the Transformer architecture, specifically adapted for attitude estimation. Unlike traditional RNNs, Attformer uses a self-attention mechanism, allowing it to weigh the importance of different parts of the input sequence when making predictions. This enables it to capture long-range dependencies in the IMU data, which is crucial for mitigating drift.
Key methodological aspects include:
- Problem Formulation: Attitude estimation is reframed as an optimization problem rather than a continuous integration process.
- Self-Attention Mechanism: The Transformer's encoder-decoder structure is employed, but with modifications for IMU data. NLP-specific components like embeddings and softmax layers are removed, and raw IMU measurements (accelerometer, gyroscope, magnetometer) are used directly as input.
- Quaternion Representation: Attitudes are represented using unit quaternions, which are mathematically more stable and computationally efficient than rotation matrices, avoiding issues like gimbal lock.
- Input and Positional Encoding: Since self-attention networks don't inherently process sequential order, positional encoding (using sine and cosine functions) is added to the input to provide temporal context.
- Encoder-Decoder Architecture: The encoder processes the input sequence, and the decoder generates the estimated attitude. Both use multi-head attention and feed-forward sub-layers.
- UKF-Attformer Hybrid: To address potential outlier estimates from pure self-attention (especially during noisy periods), a hybrid model is proposed. The UKF-Attformer incorporates the prior estimate from a traditional UKF as an additional input feature to the Attformer. This allows the model to leverage the strengths of both approaches – the robustness of UKF and the pattern-learning capabilities of self-attention.
- Baselines: The proposed methods are compared against a 2-layer Gated Recurrent Unit (GRU), a standard Unscented Kalman Filter (UKF), and the iOS CoreMotion API.
Datasets
The study utilizes a publicly available smartphone dataset collected by Chen et al. [35]. This dataset comprises:
- 158 sequences totaling over 42 km of movement.
- Data captured from five different users and four different types of smartphones.
- A variety of attachments, activities, and users to ensure broad applicability.
- IMU data collected at 100 Hz, synchronized with ground truth from a Motion Capture system.
The dataset was randomly split into training, validation, and test sets. A single sequence was held out for each user, activity, and device to ensure unseen evaluation.
Key Insights and Ideas
- Self-attention excels at long-term dependencies: Unlike RNNs, self-attention can effectively capture and utilize information from distant points in the IMU data stream, leading to more accurate and less drifted attitude estimates.
- Treating attitude estimation as an optimization problem is effective: Moving away from continuous integration and framing it as an optimization problem allows for novel deep learning approaches.
- Prior knowledge significantly improves deep learning: Combining the robust, albeit sometimes limited, estimates from a UKF with the self-attention mechanism (UKF-Attformer) dramatically reduces outlier errors and improves overall accuracy. This suggests that hybrid models can offer the best of both worlds.
- Generalizability is key: The Attformer and UKF-Attformer demonstrate superior generalizability across different users, devices, and activities compared to other methods, without requiring context-specific tuning.
- Raw IMU data is sufficient: The model successfully learns from raw accelerometer, gyroscope, and magnetometer data, eliminating the need for complex pre-processing or sensor noise modeling.
Key Results
The paper presents quantitative results across various metrics (RMSE, Angular Distance, Inner Product) and scenarios (different users, activities, devices).
- Overall Performance: The UKF-Attformer achieved the best performance, with a mean angular distance of 10.86°.
- Comparison to Baselines:
- iOS CoreMotion API: Mean angular distance of 117.31° (significantly outperformed).
- GRU: Mean angular distance of 21.90°.
- UKF: Mean angular distance of 16.38°.
- Attformer: Mean angular distance of 16.28°.
- Outlier Reduction: The UKF-Attformer effectively eliminated the large outlier estimates observed in the standalone Attformer, particularly in scenarios with significant noise spikes.
- Generalizability: The self-attention-based methods (Attformer and UKF-Attformer) showed significantly better performance across diverse users, devices, and activities compared to GRU and UKF.
Limitations and Future Work
- Limitations:
- The standalone Attformer can produce large outlier estimates, especially in the presence of significant noise spikes, due to its lack of explicit prior state knowledge.
- While the UKF-Attformer mitigates this, the computational cost of running both a UKF and a deep learning model might be a consideration for real-time embedded systems.
- Future Work:
- Further development of the framework into an end-to-end inertial odometry solution.
- Investigating ways to further optimize the hybrid UKF-Attformer for computational efficiency on resource-constrained devices.
- Exploring other self-attention variants or architectural improvements for even greater accuracy and robustness.
Summary updated