Abstract
In this paper we propose a real-time, calibration-agnostic and effective localization system for self-driving cars. Our method learns to embed the online LiDAR sweeps and intensity map into a joint deep embedding space. Localization is then conducted through an efficient convolutional matching between the embeddings. Our full system can operate in real-time at 15Hz while achieving centimeter level accuracy across different LiDAR sensors and environments. Our experiments illustrate the performance of the proposed approach over a large-scale dataset consisting of over 4000km of driving.
Introduction
Precise localization is a fundamental requirement for autonomous vehicles, typically necessitating centimeter-level accuracy. Traditional localization techniques generally rely on geometric matching (e.g., Iterative Closest Point) or intensity-based matching. Geometric methods often struggle in environments with sparse features, such as highways or tunnels. Intensity-based methods, while more descriptive, are highly sensitive to sensor calibration and environmental changes.
This work introduces a system that is calibration-agnostic and robust to environmental variations by leveraging deep learning to process LiDAR intensity data. By embedding both online sweeps and pre-built maps into a shared feature space, the system learns to focus on stable road features while ignoring sensor-specific noise.
Methodology
The core of the system is a Deep Matching Network that processes online LiDAR data and a high-definition (HD) intensity map.
Deep Matching and Joint Embedding
The architecture utilizes two deep neural networks to transform noisy online LiDAR sweeps and clean HD map data into a joint embedding space. This transformation allows the model to extract reliable features for matching, effectively bypassing the need for manual sensor calibration.
Efficient Inference
To achieve real-time performance, the system employs several key techniques:
- FFT-based Matching: The search for the optimal vehicle pose (translation and rotation) is performed in the frequency domain using Fast Fourier Transforms (FFT). This turns a computationally expensive exhaustive search into a highly efficient operation.
- Recursive Bayesian Inference: The system maintains a probability distribution (belief) over the vehicle's state, updated over time by integrating motion models (odometry) and GPS data with the LiDAR-based observations.
- Sub-pixel Accuracy: A soft-argmax operation is applied to the matching scores to achieve centimeter-level precision beyond the discrete grid resolution.
Experimental Evaluation
The system was evaluated on a massive dataset comprising over 4,000 km of driving across North American cities.
Datasets
- Highway-LidarA: 3,000+ km of data used for primary training and validation.
- Misc-LidarB: A 200 km transfer set collected with a different LiDAR model to test hardware generalization.
Key Results
- Precision: The system achieved a median error of 6–7 cm in both lateral and longitudinal directions.
- Robustness: On highway segments, the proposed method maintained a 0% failure rate (error > 1m), significantly outperforming geometric ICP methods which failed over 7% of the time.
- Generalization: Without retraining, the system maintained high accuracy (~11 cm median error) when deployed on entirely different LiDAR hardware.
- Speed: The entire pipeline operates at 15Hz, meeting the real-time constraints of autonomous driving.
Conclusion
This research demonstrates that localization can be framed as a learning problem to overcome the limitations of manual calibration and geometric sparsity. By utilizing deep embeddings and efficient frequency-domain matching, the proposed system provides a robust, real-time solution for high-precision autonomous vehicle localization across diverse environments and hardware platforms.
Summary updated