AugMapNet: Improving Spatial Latent Structure via BEV Grid Augmentation for Enhanced Vectorized Online HD Map Construction

Thomas Monninger, Md Zafar Anwar, Stanislaw Antol, Steffen Staab, Sihao Ding

TL;DR

This paper introduces AugMapNet, a novel method for constructing vectorized High-Definition (HD) maps in real-time from camera images. By augmenting the intermediate Bird's-Eye View (BEV) feature grid with dense spatial information from a predicted raster map, AugMapNet significantly improves the accuracy and structure of vectorized map predictions, outperforming existing methods.

What Problem Does This Paper Address?

Autonomous vehicles rely on accurate, real-time understanding of their static environment, often represented as vectorized HD maps. While traditional approaches use pre-built HD maps, these can become outdated due to real-world changes. Online HD map construction aims to generate these maps dynamically from sensor data. A key challenge is bridging the gap between the dense, pixel-level information captured by cameras and the sparse, instance-level representation required for vectorized maps (like lane dividers and road boundaries). Existing methods often predict intermediate raster maps and then post-process them into vectors, which can be inefficient and lose information. Alternatively, direct vector prediction methods often suffer from sparse supervision, leading to less structured latent representations. This paper addresses how to inject dense spatial supervision into the latent representation for improved vectorized map construction.

Methods

The core contribution of AugMapNet is its latent BEV grid augmentation technique. This method enhances the intermediate BEV feature grid generated by an encoder. Here's how it works:

  1. BEV Encoder: A standard BEV encoder processes camera images to create a latent BEV grid.
  2. Parallel Raster Map Decoder: A separate, lightweight decoder is trained to predict a rasterized semantic map (Mraster) from this latent BEV grid.
  3. Augmentation: The predicted raster map is then processed and added back to the original latent BEV grid, creating an augmented BEV grid (Baug). This injects dense spatial information.
  4. Gradient Stopping: Crucially, gradient stopping is applied. This means the gradients from the raster map prediction are not backpropagated to the main BEV encoder. This treats the raster map as an immutable prior, preventing it from creating a high-frequency latent space and ensuring the encoder still learns to produce features useful for vector decoding.
  5. Vector Map Decoder: The augmented BEV grid (Baug) is then fed into a standard vector map decoder to produce the final vectorized HD map.
  6. Additional BEV Processing: The architecture also includes two CNN blocks for further processing of the latent BEV grid, which helps structure the latent space and improve feature representation.

Datasets

The paper uses two prominent autonomous driving datasets:

  • nuScenes: A widely used multimodal dataset for autonomous driving tasks.
  • Argoverse2: Another large-scale dataset for autonomous driving, used to test the generalizability of the method.

The authors resample both datasets to 2 Hz for consistency and use their respective train and validation splits.

Key Insights and Ideas

  • Dense Supervision for Sparse Output: The paper argues that injecting dense spatial supervision (from a raster map) into the latent space of a vector decoder is key to improving performance. This directly addresses the inherent sparsity of supervision in vector-only methods.
  • Raster Map as an "Immutable Prior": The use of gradient stopping to treat the predicted raster map as a fixed prior is a clever way to leverage its dense supervision without negatively impacting the primary vector decoding task.
  • Structured Latent Space: Augmentation leads to a demonstrably more structured latent BEV grid, which correlates with improved prediction accuracy. This suggests that the augmentation process helps the model learn more meaningful and organized representations.
  • Generalizability: The method shows strong performance not only on its primary baseline (StreamMapNet) but also when applied to a different model (SQD-MapNet), indicating its broad applicability.

Key Results

AugMapNet achieves significant improvements:

  • nuScenes Dataset:
    • A 13.3% relative improvement in overall mAP compared to the state-of-the-art StreamMapNet baseline on a 60m perception range.
  • Substantial gains on pedestrian crossings (26.3% relative improvement).
  • Even higher improvements on larger perception ranges, reaching 24.6% on 100m.
  • Argoverse2 Dataset:
    • Outperforms baselines, achieving 58.8% mAP, confirming broader applicability.
  • Latent Space Analysis:
    • Higher Silhouette scores (indicating better clustering) and Mutual Information (MI) values (indicating higher similarity to ground truth) for AugMapNet's latent BEV grid.
  • A positive correlation between MI and vector map prediction performance is observed.
  • Efficiency: The model size and runtime (FPS) are comparable to the baseline, with only a slight increase in VRAM usage.

Limitations and Future Work

  • Limitations:
    • The paper notes that while AugMapNet improves performance, it still relies on a learned BEV encoder and a vector decoder, which are standard components. The novelty lies in the augmentation strategy.
  • The "Oracle" experiment, which uses the ground truth raster map for augmentation, achieves a very high mAP (91.4%), suggesting there's still room for improvement in the learned raster map prediction.
  • Future Work:
    • The authors suggest that their method's effectiveness on larger perception ranges and its ability to improve other baselines indicate promising directions for further research in enhancing latent BEV representations for HD map construction.
  • Further exploration of different gradient stopping strategies and BEV processing techniques could yield additional gains.

Summary updated

Abstract

Autonomous driving requires an understanding of the infrastructure elements, such as lanes and crosswalks. To navigate safely, this understanding must be derived from sensor data in real-time and needs to be represented in vectorized form. Learned Bird's-Eye View (BEV) encoders are commonly used to combine a set of camera images from multiple views into one joint latent BEV grid. Traditionally, from this latent space, an intermediate raster map is predicted, providing dense spatial supervision but requiring post-processing into the desired vectorized form. More recent models directly derive infrastructure elements as polylines using vectorized map decoders, providing instance-level information. Our approach, Augmentation Map Network (AugMapNet), proposes latent BEV grid augmentation, a novel technique that significantly enhances the latent BEV representation. AugMapNet combines vector decoding and dense spatial supervision more effectively than existing architectures while remaining as straightforward to integrate and as generic as auxiliary supervision. Experiments on nuScenes and Argoverse2 datasets demonstrate significant improvements in vectorized map prediction performance up to 13.3% over the StreamMapNet baseline on 60m range and greater improvements on larger ranges. We confirm transferability by applying our method to another baseline and find similar improvements. A detailed analysis of the latent BEV grid confirms a more structured latent space of AugMapNet and shows the value of our novel concept beyond pure performance improvement. The code will be released soon.
BibTeX
@inproceedings{monninger2026augmapnet,
  author = {Thomas Monninger and Md Zafar Anwar and Stanislaw Antol and Steffen Staab and Sihao Ding},
  title = {{AugMapNet: Improving Spatial Latent Structure via BEV Grid Augmentation for Enhanced Vectorized Online HD Map Construction}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2026},
}

Loading related content...

Figure