Multi-Sensor-Assisted Low-Cost Indoor Non-Visual Semantic Map Construction and Localization for Modern Vehicles

Guangxiao Shao, Fanyu Lin, Chao Li, Wei Shao, Wennan Chai, Xiaorui Xu, Mingyue Zhang, Zhen Sun, Qingdang LiSensors 2024 · DOI

TL;DR

This paper presents a novel, low-cost system for constructing semantic maps and localizing modern vehicles within unknown indoor environments, specifically targeting challenging scenarios like underground parking. By fusing data from common vehicle sensors and identifying "non-visual semantic landmarks" (like entrances and road junctions), the system achieves accurate localization without relying on expensive LiDAR or cameras.

What Problem Does This Paper Address?

Accurate indoor localization for vehicles is becoming increasingly crucial with the rise of intelligent transportation systems, autonomous driving features, and in-vehicle infotainment. Traditional indoor mapping methods are often manual, costly, and time-consuming. Existing sensor-based approaches, particularly those relying on LiDAR or cameras, can be expensive and struggle with dynamic indoor environments (e.g., moving pedestrians, vehicles, changing lighting). This paper aims to bridge this gap by proposing a versatile, low-cost solution that leverages readily available sensors on modern vehicles to create semantic maps and enable robust localization in previously unmapped indoor spaces.

Methods

The proposed system comprises three main stages:

  1. Non-Visual Semantic Landmark Detection: This is a core innovation. Instead of relying on visual cues, the system identifies "non-visual semantic landmarks" such as entrances/exits, ramp entrances/exits, and road nodes (intersections). These are detected using a sliding window approach on sensor data (IMU, GNSS, light sensor, odometer). Key features and their changes at window joint points are analyzed using a Spark logistic regression model. For road nodes, changes in heading angle, trajectory curvature, and triangle aspect ratio are used.
  2. Single-Trajectory Semantic Map Construction: Once landmarks are detected, a lightweight semantic map is built. This map consists of:
  3. Waypoints: Points along the vehicle's trajectory.
  4. Non-Visual Semantic Landmarks: The identified landmarks.
  5. Wi-Fi Fingerprints: Received Signal Strength (RSS) of Wi-Fi access points. A sliding window-based fusion method is used to enhance the stability of Wi-Fi fingerprints. Associations are made between waypoints and Wi-Fi fingerprints via UTC, and landmarks are linked to waypoints by mileage.

  6. Landmark Matching-Based Localization with Graph Optimization: To estimate the vehicle's position within the constructed semantic map, a two-phase localization method is employed:

  7. Initial Matching: This phase prioritizes matching non-visual semantic landmarks by type. If landmarks are insufficient, Wi-Fi fingerprinting is used. A "landmark matching quality" score is calculated based on distances of matched landmarks and Wi-Fi fingerprints.
  8. Secondary Matching: This phase focuses on non-visual semantic landmark matching, leveraging the correlation between landmarks and previously matched ones, considering height, angle, and distance. A graph optimization approach is then applied to refine the vehicle's pose by minimizing a loss function, effectively aligning the trajectory map with the venue map and reducing location errors.

Datasets

The paper does not explicitly mention the use of pre-existing public datasets for training or evaluation. Instead, the authors conducted field experiments in two real-world shopping mall scenes in Shanghai. They collected data using a specific vehicle platform (AITO M5 Standard Edition) equipped with its onboard sensors. The venue maps were constructed from the vehicle's trajectory data, and the ground truth for evaluation was derived from video image timestamps and comparisons with existing mapping services (AMAP).

Key Insights and Ideas

  • "Non-Visual Semantic Landmarks" are Key: The concept of identifying landmarks that are not visually apparent (like entrances, exits, road junctions) is a significant departure from traditional visual SLAM. This allows the system to function in environments where visual data is unreliable or unavailable.
  • Low-Cost Sensor Fusion is Viable: The paper demonstrates that by intelligently fusing data from common, low-cost sensors (IMU, odometer, Wi-Fi, GNSS, light sensor), robust indoor localization is achievable, making it accessible for a wider range of vehicles.
  • Sliding Window for Dynamic Environments: The sliding window approach for landmark detection is crucial for handling the dynamic nature of indoor environments, allowing the system to adapt to changing conditions and extract relevant features over time.
  • Graph Optimization for Precision: The integration of graph optimization significantly enhances localization accuracy by iteratively refining the vehicle's pose based on the relationships between map components.
  • Wi-Fi Fingerprint Fusion for Stability: Using a sliding window fusion method for Wi-Fi fingerprints helps to mitigate the impact of signal fluctuations and interference, leading to more reliable localization.

Key Results

  • Non-Visual Semantic Landmark Detection: Achieved a high accuracy of 98.1% (implied by a low false rate of 1.90% and miss rate of 7.61% across both malls).
  • Localization Accuracy: The proposed landmark matching-based localization method achieved an average localization error of 1.41 m in mall 1 and 1.34 m in mall 2.
  • Improvement over Wi-Fi Fingerprinting: The proposed method showed a significant improvement over a pure Wi-Fi fingerprinting-based localization method, with an average error of 2.62 m in mall 1 and 2.62 m in mall 2. This represents an improvement of 59.64% in mall 2.

Limitations and Future Work

  • Single-Trajectory Map Coverage: The authors acknowledge that a single-trajectory semantic map may not cover the entire indoor scene.
  • Crowdsourced Mapping: To address the coverage limitation, they propose fusing multiple single-trajectory maps from different vehicles to build a comprehensive, crowdsourced indoor non-visual semantic map. This is identified as a future research direction.
  • Enhanced Wi-Fi Fingerprint Maturity: Crowdsourced mapping is also expected to improve the maturity of Wi-Fi fingerprints and further reduce detection and localization errors.
  • Generalizability: While tested in shopping malls, the system's performance in other types of indoor environments with different layouts and dynamics would be an interesting area for further investigation.

Summary updated

Abstract

With the transformation and development of the automotive industry, low-cost and seamless indoor and outdoor positioning has become a research hotspot for modern vehicles equipped with in-vehicle infotainment systems, Internet of Vehicles, or other intelligent systems (such as Telematics Box, Autopilot, etc.). This paper analyzes modern vehicles in different configurations and proposes a low-cost, versatile indoor non-visual semantic mapping and localization solution based on low-cost sensors. Firstly, the sliding window-based semantic landmark detection method is designed to identify non-visual semantic landmarks (e.g., entrance/exit, ramp entrance/exit, road node). Then, we construct an indoor non-visual semantic map that includes the vehicle trajectory waypoints, non-visual semantic landmarks, and Wi-Fi fingerprints of RSS features. Furthermore, to estimate the position of modern vehicles in the constructed semantic maps, we proposed a graph-optimized localization method based on landmark matching that exploits the correlation between non-visual semantic landmarks. Finally, field experiments are conducted in two shopping mall scenes with different underground parking layouts to verify the proposed non-visual semantic mapping and localization method. The results show that the proposed method achieves a high accuracy of 98.1% in non-visual semantic landmark detection and a low localization error of 1.31 m.
BibTeX
@inproceedings{shao2024multi,
  author = {Guangxiao Shao and Fanyu Lin and Chao Li and Wei Shao and Wennan Chai and Xiaorui Xu and Mingyue Zhang and Zhen Sun and Qingdang Li},
  title = {{Multi-Sensor-Assisted Low-Cost Indoor Non-Visual Semantic Map Construction and Localization for Modern Vehicles}},
  booktitle = {Sensors},
  year = {2024},
  doi = {10.3390/s24134263},
}
Figure