Abstract: In this paper we propose a real-time, calibration-agnostic and effective localization system for self-driving cars. Our method learns to embed the online LiDAR sweeps and intensity map into a joint feature space, enabling precise matching that overcomes the limitations of traditional geometric and manual calibration methods.
Introduction
Accurate localization is a fundamental requirement for autonomous driving, often referred to as the "where am I?" problem. While standard GPS provides meter-level accuracy, self-driving vehicles require centimeter-level precision to safely navigate lanes and complex urban intersections. This paper introduces a deep-learning-based localization system that achieves this precision in real-time (15Hz) by leveraging LiDAR intensity maps and online sweeps.
The Localization Challenge
Traditional geometric methods, such as Iterative Closest Point (ICP), rely on structural features to align sensor data with a map. However, these methods often fail in "featureless" environments like long tunnels or open highways where geometric segments appear identical.
LiDAR intensity maps, which record the reflectivity of surfaces (e.g., road paint vs. asphalt), provide a much richer signal for localization. The primary drawback is that intensity values are highly sensor-dependent; values vary by manufacturer, beam configuration, and even weather conditions. Historically, this necessitated laborious manual calibration for every vehicle. The authors address this by creating a calibration-agnostic system that learns to identify similarities between a map and a live sensor feed regardless of raw intensity discrepancies.
Proposed Methodology
The authors frame localization as a Deep Recursive Bayesian Inference problem. The architecture centers on a "Matching Network" designed for speed and robustness:
- Joint Embedding: Two deep convolutional neural networks (CNNs) process the data in parallel. One embeds the pre-built HD intensity map, while the other embeds the live online LiDAR sweeps. These networks transform raw data into a common feature space where matching is optimized.
- Convolutional Matching: To determine the vehicle's position, the system searches over a 3-DoF pose (2D position and rotation). The online embedding acts as a "kernel" that is convolved over the map embedding.
- The FFT Trick: To maintain real-time performance, this convolution is performed in the Fourier domain using Fast Fourier Transforms (FFT). This reduces computational complexity, allowing the search to conclude in milliseconds.
- Soft-Argmax: Instead of selecting a single best-scoring pixel, the system uses a "soft-argmax" to achieve sub-pixel accuracy and provide a better representation of uncertainty.
Performance and Generalization
The system was evaluated using a massive real-world dataset comprising over 4,000km of driving data across North American cities. A key strength of the model is its ability to utilize both geometric and intensity-based features simultaneously.
Key Results
- Accuracy: The system achieved a median total error of approximately 7cm on the primary test set.
- Robustness: While ICP failure rates (error > 1m) exceeded 7% in some sequences, this method maintained a failure rate as low as 0.35%.
- Zero-Shot Transfer: In a test of generalization, a model trained on one LiDAR sensor type was tested on a completely different sensor. It maintained a median error of ~11.7cm, proving it can handle significant domain shifts without retraining.
- Speed: The full system operates at 15Hz, with the matching step taking only ~1.4ms on a standard GPU.
Conclusion and Future Work
By learning the calibration implicitly through deep embeddings, the proposed system offers a "plug-and-play" solution for autonomous vehicle localization. It avoids the local minima traps of ICP by searching the entire discretized space to find a global optimum.
Current limitations include a constraint to 3-DoF (assuming a 2D plane), which does not account for pitch, roll, or vertical changes. Future work will explore expanding the system to 6-DoF poses and integrating more temporal information to further enhance the smoothness and reliability of the localization results.
Summary updated