Abstract
Autonomous driving requires a real-time understanding of infrastructure elements like lanes and crosswalks, ideally represented in vectorized form. While Bird's-Eye View (BEV) encoders are standard for fusing multi-view camera data, a fundamental trade-off exists between rasterized decoding (dense supervision but slow post-processing) and vectorized decoding (efficient but sparse supervision). We present AugMapNet, which introduces Latent BEV Grid Augmentation. This technique enhances the latent BEV representation by effectively combining vector decoding with dense spatial supervision. Experiments on nuScenes and Argoverse2 datasets demonstrate significant performance gains—up to 13.3% over the StreamMapNet baseline at a 60m range, with even greater improvements at longer distances. Analysis confirms that AugMapNet produces a more structured latent space, bridging the gap between rasterized and vectorized mapping paradigms.
Introduction
To navigate safely, autonomous vehicles require High-Definition (HD) maps containing precise geometries of lanes, dividers, and crosswalks. Because environments are dynamic, these maps must often be generated "online" from onboard sensors. Current research follows two main paradigms: rasterized decoding, which provides dense supervision but requires heuristic-heavy post-processing, and vectorized decoding, which is planning-ready but often suffers from sparse supervision and a poorly structured latent space. AugMapNet addresses this by asking: Can we leverage the rich learning signal of a raster map without losing the efficiency of a vectorized model?
Methodology: Latent BEV Grid Augmentation
The core innovation of AugMapNet is the integration of raster information directly back into the model's latent feature grid. Unlike traditional auxiliary tasks that merely add a loss function, AugMapNet uses a feedback loop to "augment" the model's internal representation.
As shown in the architecture above, the model first generates a standard BEV feature grid from camera images. A small raster decoder predicts a semantic segmentation map, and these features are then encoded and added back into the original BEV grid. To maintain stability, the authors employ gradient stopping, treating the predicted raster map as an immutable prior. This prevents the vector-prediction task from interfering with the semantic feature learning. Finally, two additional CNN blocks process the augmented grid to smooth and integrate the features before they reach the vector decoder.
Experimental Evaluation
The model was rigorously tested on the nuScenes and Argoverse2 datasets. To ensure the model learned generalizable spatial features rather than memorizing specific locations, the authors used geospatially disjoint splits.
Key Performance Metrics
- Accuracy Boost: On nuScenes (60m range), AugMapNet achieved 38.3% mAP, a significant improvement over the 33.8% baseline.
- Long-Range Robustness: One of the most striking findings was that AugMapNet’s relative improvement increases at longer ranges. At 100m, it showed a 24.6% improvement, suggesting that the added spatial structure acts as a vital guide when sensor data is sparse or blurry.
- Complex Element Recognition: Crosswalks, which are spatially complex, saw a 26.3% relative improvement, benefiting from the dense area-based supervision provided by the raster augmentation.
Latent Space Analysis and Qualitative Results
Beyond raw metrics, the authors investigated why the model performs better. Using Principal Component Analysis (PCA) to visualize the latent BEV grid, they found that AugMapNet’s latent space is significantly more organized.
In the figure above, the PCA components (PC 1-3) for AugMapNet clearly reflect the underlying road geometry, whereas the baseline (StreamMapNet) appears more chaotic. This structured latent space makes the final task of the vector decoder—identifying and connecting points into lines—much more straightforward.
Conclusion and Future Work
AugMapNet demonstrates that dense spatial supervision can be successfully integrated into vectorized mapping pipelines via latent grid augmentation. The method is modular, showing consistent gains when applied to different baselines like StreamMapNet and SQD-MapNet, and remains real-time capable at 11.9 FPS.
Future research will focus on closing the "Oracle gap"—the performance difference between using predicted rasters versus ground-truth rasters—and exploring more sophisticated fusion techniques, such as cross-attention, to further refine how semantic priors are integrated into the BEV representation.
Summary updated