AugMapNet: Improving Spatial Latent Structure via BEV Grid Augmentation for Enhanced Vectorized Online HD Map Construction

Thomas Monninger, Md Zafar Anwar, Stanislaw Antol, Steffen Staab, Sihao Ding

Abstract

Autonomous driving requires a real-time understanding of infrastructure elements like lanes and crosswalks, ideally represented in vectorized form. While Bird's-Eye View (BEV) encoders are standard for fusing multi-view camera data, a fundamental trade-off exists between rasterized decoding (dense supervision but slow post-processing) and vectorized decoding (efficient but sparse supervision). We present AugMapNet, which introduces Latent BEV Grid Augmentation. This technique enhances the latent BEV representation by effectively combining vector decoding with dense spatial supervision. Experiments on nuScenes and Argoverse2 datasets demonstrate significant performance gains—up to 13.3% over the StreamMapNet baseline at a 60m range, with even greater improvements at longer distances. Analysis confirms that AugMapNet produces a more structured latent space, bridging the gap between rasterized and vectorized mapping paradigms.

Introduction

To navigate safely, autonomous vehicles require High-Definition (HD) maps containing precise geometries of lanes, dividers, and crosswalks. Because environments are dynamic, these maps must often be generated "online" from onboard sensors. Current research follows two main paradigms: rasterized decoding, which provides dense supervision but requires heuristic-heavy post-processing, and vectorized decoding, which is planning-ready but often suffers from sparse supervision and a poorly structured latent space. AugMapNet addresses this by asking: Can we leverage the rich learning signal of a raster map without losing the efficiency of a vectorized model?

Methodology: Latent BEV Grid Augmentation

The core innovation of AugMapNet is the integration of raster information directly back into the model's latent feature grid. Unlike traditional auxiliary tasks that merely add a loss function, AugMapNet uses a feedback loop to "augment" the model's internal representation.

The AugMapNet architecture features a BEV Encoder, a raster-based augmentation loop, and a final Vector Map Decoder.

As shown in the architecture above, the model first generates a standard BEV feature grid from camera images. A small raster decoder predicts a semantic segmentation map, and these features are then encoded and added back into the original BEV grid. To maintain stability, the authors employ gradient stopping, treating the predicted raster map as an immutable prior. This prevents the vector-prediction task from interfering with the semantic feature learning. Finally, two additional CNN blocks process the augmented grid to smooth and integrate the features before they reach the vector decoder.

Experimental Evaluation

The model was rigorously tested on the nuScenes and Argoverse2 datasets. To ensure the model learned generalizable spatial features rather than memorizing specific locations, the authors used geospatially disjoint splits.

Key Performance Metrics

  • Accuracy Boost: On nuScenes (60m range), AugMapNet achieved 38.3% mAP, a significant improvement over the 33.8% baseline.
  • Long-Range Robustness: One of the most striking findings was that AugMapNet’s relative improvement increases at longer ranges. At 100m, it showed a 24.6% improvement, suggesting that the added spatial structure acts as a vital guide when sensor data is sparse or blurry.
  • Complex Element Recognition: Crosswalks, which are spatially complex, saw a 26.3% relative improvement, benefiting from the dense area-based supervision provided by the raster augmentation.

Latent Space Analysis and Qualitative Results

Beyond raw metrics, the authors investigated why the model performs better. Using Principal Component Analysis (PCA) to visualize the latent BEV grid, they found that AugMapNet’s latent space is significantly more organized.

Qualitative comparison and PCA visualization of the latent BEV grid. AugMapNet (right) shows a more structured latent space and more accurate map predictions compared to the baseline.

In the figure above, the PCA components (PC 1-3) for AugMapNet clearly reflect the underlying road geometry, whereas the baseline (StreamMapNet) appears more chaotic. This structured latent space makes the final task of the vector decoder—identifying and connecting points into lines—much more straightforward.

Conclusion and Future Work

AugMapNet demonstrates that dense spatial supervision can be successfully integrated into vectorized mapping pipelines via latent grid augmentation. The method is modular, showing consistent gains when applied to different baselines like StreamMapNet and SQD-MapNet, and remains real-time capable at 11.9 FPS.

Future research will focus on closing the "Oracle gap"—the performance difference between using predicted rasters versus ground-truth rasters—and exploring more sophisticated fusion techniques, such as cross-attention, to further refine how semantic priors are integrated into the BEV representation.

Summary updated

Abstract

Autonomous driving requires an understanding of the infrastructure elements, such as lanes and crosswalks. To navigate safely, this understanding must be derived from sensor data in real-time and needs to be represented in vectorized form. Learned Bird's-Eye View (BEV) encoders are commonly used to combine a set of camera images from multiple views into one joint latent BEV grid. Traditionally, from this latent space, an intermediate raster map is predicted, providing dense spatial supervision but requiring post-processing into the desired vectorized form. More recent models directly derive infrastructure elements as polylines using vectorized map decoders, providing instance-level information. Our approach, Augmentation Map Network (AugMapNet), proposes latent BEV grid augmentation, a novel technique that significantly enhances the latent BEV representation. AugMapNet combines vector decoding and dense spatial supervision more effectively than existing architectures while remaining as straightforward to integrate and as generic as auxiliary supervision. Experiments on nuScenes and Argoverse2 datasets demonstrate significant improvements in vectorized map prediction performance up to 13.3% over the StreamMapNet baseline on 60m range and greater improvements on larger ranges. We confirm transferability by applying our method to another baseline and find similar improvements. A detailed analysis of the latent BEV grid confirms a more structured latent space of AugMapNet and shows the value of our novel concept beyond pure performance improvement. The code will be released soon.
BibTeX
@inproceedings{monninger2026augmapnet,
  author = {Thomas Monninger and Md Zafar Anwar and Stanislaw Antol and Steffen Staab and Sihao Ding},
  title = {{AugMapNet: Improving Spatial Latent Structure via BEV Grid Augmentation for Enhanced Vectorized Online HD Map Construction}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2026},
}
Figure