Abstract
In this paper, we show that High-Definition (HD) maps provide strong priors that can boost the performance and robustness of modern 3D object detectors. We design a single-stage detector, HDNET, that extracts geometric and semantic features from HD maps. To address scenarios where maps are unavailable, we also propose a map prediction module that estimates the map on the fly from raw LiDAR data. Extensive experiments on KITTI and a large-scale 1-million-frame benchmark demonstrate that the proposed map-aware detector consistently outperforms state-of-the-art methods in both mapped and unmapped scenarios, while maintaining a real-time speed of 20 frames per second.
Introduction
Autonomous vehicles (AVs) rely heavily on LiDAR for 3D object detection, yet this technology faces significant challenges. LiDAR point clouds become increasingly sparse at long ranges, making it difficult to distinguish distant vehicles from background noise. Furthermore, standard detectors often assume a flat ground plane, but real-world road slopes can cause vertical shifts in point positions—a 1-degree incline can shift a point by over a meter at 70 meters—confusing 2D-convolutional detectors that treat height as a fixed feature channel.
While HD maps are ubiquitous in motion planning, they are frequently underutilized in perception. HDNET bridges this gap by incorporating "prior knowledge" from maps—specifically ground geometry and road semantics—directly into the detection pipeline to improve accuracy and robustness.
Methodology
HDNET operates on a Bird’s Eye View (BEV) representation of LiDAR data. The architecture integrates three primary innovations to leverage map data effectively:
- Geometric Prior (Ground Normalization): Instead of using absolute height ($z$), the network uses height relative to the ground ($z - z_{ground}$) provided by the HD map. This normalization "flattens" the terrain, ensuring feature consistency across varying road slopes.
- Semantic Prior (Road Mask): The road layout is rasterized into a binary channel and concatenated with LiDAR features. This spatial prior acts as an attention mechanism, focusing the model on regions where vehicles are likely to be located.
- Online Map Prediction: Recognizing that HD maps are not available everywhere, the authors designed a U-Net-based module that estimates ground height and road masks directly from a single LiDAR sweep.
To ensure the detector remains robust when maps are missing, the authors employ "map dropout" during training, randomly feeding the network empty maps so it learns to rely on raw LiDAR when necessary.
Experimental Evaluation
The researchers evaluated HDNET on the massive TOR4D dataset (containing over 1 million frames) and the public KITTI benchmark.
Performance Gains
On the TOR4D dataset, HDNET achieved a significant Average Precision (AP) gain of 5.49% for long-range objects (50-70m) when offline HD maps were available. On the KITTI benchmark, which lacks HD maps, HDNET utilized its online map prediction module (trained on TOR4D) to achieve a 2.87% AP gain in the "moderate" category, outperforming contemporary methods that used both LiDAR and camera inputs.
Key Insights
- Real-Time Efficiency: The entire pipeline runs at 20 FPS, making it suitable for real-time deployment on autonomous vehicles.
- Generalization: A map predictor trained on North American data (TOR4D) generalized effectively to European data (KITTI), providing a performance boost even without local map data.
- Long-Range Specialist: The benefits of HD map priors grow as LiDAR data degrades. The further away an object is, the more the network relies on the map prior to make a confident detection.
Conclusion
HDNET demonstrates that HD maps are a powerful resource for 3D perception. By normalizing geometric features and providing semantic context, the model achieves superior detection performance, particularly at long distances. While online map prediction helps mitigate the lack of map availability, future work may address the challenges of dynamic environments and potential map inaccuracies in rapidly changing urban landscapes.
Summary updated