Abstract
High-precision localization is a fundamental requirement for autonomous driving, yet traditional methods often rely on expensive Lidar sensors or massive 3D point-cloud maps that are difficult to scale. This paper proposes a robust self-localization system that utilizes a single monocular camera and a lightweight "compact semantic map." By representing the environment through geometric primitives—such as lane lines, lamp poles, and traffic signs—the system achieves sub-meter accuracy (0.345m RMS) while requiring significantly less storage (approximately 4KB for 270m of road). This approach offers a cost-effective and scalable solution for high-precision urban localization.
Introduction
While Differential GPS (DGPS) with Real-Time Kinematic (RTK) data provides high accuracy in open environments, it frequently fails in "urban canyons" due to signal blockage and multipath reflections. Map-based localization has emerged as a reliable alternative, yet it typically faces two major hurdles: the high cost of sensors like Lidar and the massive storage requirements of dense 3D point clouds. This research addresses these challenges by developing a localization pipeline that treats the problem as a 3D-to-2D registration between a monocular image and a vectorized semantic map.
System Architecture
The proposed system is structured into three primary modules: map management, image processing, and localization. The architecture is designed to bridge the gap between low-cost hardware and high-precision requirements.
- Compact Semantic Map: Instead of raw points, the map stores landmarks as geometric primitives. Line landmarks (lanes, poles) and point landmarks (signs) are stored as vectorized data, drastically reducing storage needs.
- Semantic-Geometric Feature Extraction: The system identifies semantic regions in camera frames and extracts precise geometric features for matching.
- Robust Pose Estimation: The vehicle's 6-DOF pose is estimated by minimizing the re-projection error between 3D map landmarks and 2D image features.
Feature Extraction and Map Representation
To match the real-world environment to the map, the system first performs semantic segmentation using PSPNet. This deep neural network identifies pixels belonging to specific classes like lanes, poles, and traffic signs. Following segmentation, RANSAC-based line fitting and region-growing techniques are applied to extract geometric primitives (lines and centroids) from the noisy segmentation masks.
This abstraction from raw pixels to geometric features is what allows the map to remain compact. For a 270-meter test stretch, the resulting map file was only 4KB, orders of magnitude smaller than traditional down-sampled Lidar maps.
Robust Pose Estimation
The core of the localization is a multi-modal re-projection error minimization. The system projects the 3D landmarks from the compact map onto the 2D image plane based on an initial pose estimate. The distance between these projections and the detected image features forms the cost function.
To ensure stability in urban environments, the authors introduced "soft constraints." By assuming the vehicle remains on relatively flat ground, constraints are added for camera height, pitch, and roll. This helps the optimization converge even when landmarks are sparse. Furthermore, a heuristic "hypothesis association" method ensures correct matching between map landmarks and image features, even with a rough initial pose guess.
Experimental Results
The method was evaluated using Sequence 04 of the KITTI Odometry Dataset. The semantic segmentation network was pre-trained on the Cityscapes dataset to ensure robust recognition.
- Accuracy: The system achieved an RMS positioning error of 0.345 meters, with 90% of errors falling below 0.5 meters.
- Angular Precision: The average orientation error was remarkably low at 0.019 radians.
- Efficiency: The lateral and vertical accuracy were found to be higher than longitudinal accuracy, a result of the geometric "steepness" of lane lines and poles in the cost function.
Compared to other state-of-the-art methods, this approach proved competitive with Lidar-based systems while using significantly less complex data structures.
Conclusion and Future Work
This research demonstrates that sub-meter localization is achievable using a single camera and a highly compressed semantic map. While the system performs well in typical urban settings, future work will focus on improving robustness in areas with sparse landmarks or heavy shadows. Integrating temporal filters, such as Kalman Filters, and expanding the model to handle sharp curves are key next steps for making this technology ready for mass-market autonomous vehicles.
Summary updated