An Integrated LSTM-Rule-Based Fusion Method for the Localization of Intelligent Vehicles in a Complex Environment

Quan Yuan, Fuwu Yan, Zhishuai Yin, Chen Lv, Jie Hu, Yue Li, Jinhai WangSensors 2024 · DOI

TL;DR

This paper introduces a novel localization system for intelligent vehicles that fuses GPS, LiDAR SLAM, and an odometer model using a combination of fuzzy logic and a Dual-LSTM network. This approach significantly improves localization accuracy and robustness in complex environments, achieving centimeter-level precision and reducing RMSE by 66% compared to EKF.

What Problem Does This Paper Address?

Accurate and reliable localization is a cornerstone of autonomous driving, enabling safe path planning and trajectory tracking. However, in complex environments, traditional localization methods like GPS and LiDAR SLAM often struggle due to signal blockage, interference, and dynamic obstacles. This leads to significant localization errors, potentially causing vehicles to deviate from their paths or even collide. The paper aims to address this challenge by developing a robust multi-source fusion method that can maintain high localization accuracy even when individual sensors are unreliable.

Methods

The proposed method integrates three key components:

  • Fuzzy Fusion-Based Localization Module: This module addresses the unreliability of individual sensors by introducing a rule-based system to dynamically assess the confidence of GPS and SLAM outputs. Fuzzy rules are used to analyze localization deviation and confidence factors, generating a weight (W1) that dictates the contribution of each sensor to the initial fused localization. This allows the system to prioritize more reliable sensor data.
  • Odometer Movement Track Matching Module: Recognizing the high accuracy of odometer data over short distances, this module uses IMU-based odometry to generate a projected localization trajectory. This projected trajectory is then used to "inhibit" noise from the initial fused localization and refine the position through trajectory matching. When GPS and SLAM drift significantly, this module combines the previous optimal rotation-translation matrix with the odometer-fitted trajectory for localization.
  • Trajectory Prediction Module (Dual-LSTM): To further enhance robustness, a Dual-LSTM network is employed for trajectory prediction. This network takes historical trajectory data and predicts future vehicle motion. This predicted trajectory can guide the vehicle when other localization methods fail and also helps in updating short-distance localization information. Additionally, a lidar-based "electronic fence" is used for safety. The Dual-LSTM architecture allows for dynamic adjustment of initial fusion localization weights based on prediction errors.

Datasets

The dataset was collected from a real vehicle platform (SV 1.0 plus) between May 2021 and October 2021. It includes:

  • GPS localization data
  • SLAM localization data
  • Data from the proposed method (initial fusion and projected localization)
  • Vehicle chassis data transmitted via CAN line
  • Vehicle driving intent identification and labeling

The dataset contains nearly 1.2 million vehicle track points sampled at 50 Hz. For training and testing the trajectory prediction model, approximately 7000 sample fragments were extracted, with the first 5700 used for training and the remaining 1300 for testing. The paper also notes that a higher precision integrated inertial navigation system was used to record ground truth information.

Key Insights and Ideas

  • Fuzzy Logic for Sensor Confidence: The paper highlights the critical need to dynamically assess sensor reliability. Using fuzzy rules to weigh GPS and SLAM inputs based on their confidence factors is a clever way to handle sensor unreliability without discarding data entirely.
  • Leveraging Odometer for Short-Term Accuracy: The insight that odometer data is highly accurate over short distances is effectively utilized. By using the shape of the projected trajectory to filter noise and match against actual trajectories, the system gains a significant accuracy boost in challenging scenarios.
  • Dual-LSTM for Predictive Robustness: The use of a Dual-LSTM network for trajectory prediction is a powerful addition. It not only provides a fallback localization mechanism but also allows for dynamic adjustment of sensor fusion weights, creating a more adaptive and resilient system.
  • Cost-Effective Solution: The method emphasizes using existing in-vehicle sensors, avoiding the need for additional hardware, which is crucial for practical deployment in autonomous vehicles with limited computational resources.

Key Results

  • Reduced RMSE: The proposed method achieved a significant reduction in the average root mean square error (RMSE) of localization by 66% compared to the EKF fusion localization method.
  • Centimeter-Level Accuracy: The system demonstrated centimeter-level localization accuracy.
  • Max Error Reduction: In real-world tests, the maximum localization error of the proposed method was 0.78 m, significantly lower than GPS (1.55 m), NDT (1.23 m), and EKF (1.12 m).
  • Improved Trajectory Prediction: The self-vehicle trajectory prediction model showed higher accuracy than the initial fusion localization, with a maximum error of no more than 0.6 m.

Limitations and Future Work

  • Not a Complete Solution: The authors acknowledge that the proposed method improves robustness and accuracy in severe environments but does not "totally" solve the localization problem. Further research is needed to address remaining challenges.
  • Computational Capacity: While the method is designed for limited computational capacity, the paper mentions that the Dual-LSTM network, though simpler than RL, still requires careful consideration for real-time implementation.
  • Future Work: The paper suggests that further study is needed to fully solve the localization problem. Specific areas for future work are not explicitly detailed but implied by the limitations. The authors also mention that the method can be applied to platforms with low computational capacity, freeing up resources for other algorithms.

Summary updated

Abstract

To improve the accuracy and robustness of autonomous vehicle localization in a complex environment, this paper proposes a multi-source fusion localization method that integrates GPS, laser SLAM, and an odometer model. Firstly, fuzzy rules are constructed to accurately analyze the in-vehicle localization deviation and confidence factor to improve the initial fusion localization accuracy. Then, an odometer model for obtaining the projected localization trajectory is constructed. Considering the high accuracy of the odometer's projected trajectory within a short distance, we used the shape of the projected localization trajectory to inhibit the initial fusion localization noise and used trajectory matching to obtain an accurate localization. Finally, the Dual-LSTM network is constructed to predict the localization and build an electronic fence to guarantee the safety of the vehicle while also guaranteeing the updating of short-distance localization information of the vehicle when the above-mentioned fusion localization is unreliable. Under the limited arithmetic condition of the vehicle platform, accurate and reliable localization is realized in a complex environment. The proposed method was verified by long-time operation on the real vehicle platform, and compared with the EKF fusion localization method, the average root mean square error of localization was reduced by 66%, reaching centimeter-level localization accuracy.
BibTeX
@inproceedings{yuan2024integrated,
  author = {Quan Yuan and Fuwu Yan and Zhishuai Yin and Chen Lv and Jie Hu and Yue Li and Jinhai Wang},
  title = {{An Integrated LSTM-Rule-Based Fusion Method for the Localization of Intelligent Vehicles in a Complex Environment}},
  booktitle = {Sensors},
  year = {2024},
  doi = {10.3390/s24124025},
}
Figure