Bi-Mapper: Holistic BEV Semantic Mapping for Autonomous Driving

Siyu Li, Kailun Yang, Hao Shi, Jiaming Zhang, Jiacheng Lin, Zhifeng Teng, Zhiyong Li

Abstract

Generating accurate Bird’s-Eye-View (BEV) semantic maps from monocular or multi-view camera inputs is a critical task for autonomous navigation. Current methodologies generally fall into two categories: those relying on geometric priors like Inverse Perspective Mapping (IPM) and those utilizing deep learning-based transformers. While the former offers local precision, it suffers from distant geometric distortion; the latter provides global context but lacks inherent geometric constraints. This paper presents Bi-Mapper, a dual-stream framework that harmonizes these approaches through a novel asynchronous mutual learning strategy and across-space loss, achieving state-of-the-art performance on major benchmarks.

Introduction

In the field of autonomous driving, the ability to transform standard front-facing camera perspectives into a top-down BEV map is essential for path planning and obstacle avoidance. However, the industry faces a dichotomy in view transformation techniques. Geometric priors, such as IPM, use camera calibration to project pixels onto a ground plane, providing high accuracy for nearby features but resulting in significant blurring and stretching for distant objects. Conversely, learning-based transformers attempt to implicitly learn the 2D-to-3D relationship. While powerful, these models often suffer from slow convergence and a lack of grounding in physical reality.

Bi-Mapper addresses this gap by fusing local geometric accuracy with global contextual learning, ensuring that the strengths of one paradigm compensate for the weaknesses of the other.

Methodology

Dual-Stream Architecture

Bi-Mapper utilizes two distinct streams to process visual information:

  1. Local-self View (LV) Stream: This stream utilizes IPM as a geometric prior. It translates input images into the camera coordinate system, performs semantic segmentation via a U-Net architecture, and subsequently projects the results into the ego-vehicle coordinate system. This ensures that local features remain sharp and geometrically consistent.
  2. Global-cross View (GV) Stream: This stream employs Multi-Layer Perceptrons (MLPs) to learn a global transformation directly from pixel coordinates to the ego coordinate system. This allows the network to infer depth implicitly and maintain a holistic understanding of the entire scene.

Key Innovations

To effectively integrate these streams, the authors introduced two specific mechanisms:

  • Across-Space Loss (ASL): To mitigate the geometric distortions inherent in IPM, ASL applies a supervision loss specifically within the camera coordinate system. This forces the LV stream to correct for the "stretching" of distant objects before they are fused into the final BEV map.
  • Asynchronous Mutual Learning (AML): Recognizing that the geometric stream (LV) is inherently more accurate during the early stages of training, AML adopts an asynchronous strategy. Initially, the LV stream acts as a "teacher" to the GV stream. Once the GV stream matures and gains contextual stability, the two streams transition into a mutual exchange of knowledge, refining each other's predictions.

Experimental Results

Datasets and Evaluation

The framework was rigorously tested on two primary benchmarks:

  • nuScenes: A large-scale real-world dataset featuring six camera views and complex urban environments.
  • Cam2BEV: A synthetic dataset designed for BEV mapping with four camera views.
  • Real-world Deployment: The model was also validated on a custom four-wheel transport robot in industrial and campus settings to ensure generalization beyond curated datasets.

Performance Metrics

Bi-Mapper demonstrated superior performance compared to existing state-of-the-art models:

  • nuScenes: Achieved a 37.9% IoU (Intersection over Union), outperforming BEVFormer (35.8%) and HDMapNet (32.9%).
  • Cam2BEV: Reached 86.7% IoU, marking a 4.1% improvement over contemporary methods.
  • Efficiency: The model maintains an inference time of approximately 2.4 seconds, making it competitive and viable for real-time autonomous applications.

Ablation Studies

Ablation experiments confirmed the necessity of the proposed modules. Removing either the AML or ASL components resulted in a significant performance degradation (approximately 1-3% IoU), highlighting that the interaction between the streams is as vital as the dual-stream architecture itself.

Conclusion and Future Work

Bi-Mapper successfully bridges the gap between geometric priors and global learning-based transformations. By leveraging the "teacher" role of geometry early in training and enforcing spatial consistency through ASL, the model produces BEV maps that are both locally precise and globally consistent.

Future research will focus on addressing the remaining blurriness of extremely distant objects and incorporating temporal information from video sequences to improve the stability of dynamic object detection. Additionally, the authors aim to explore the integration of LiDAR and Radar data into the dual-stream framework to create a more robust multi-modal mapping system.

Summary updated

BibTeX
@article{li2023bi,
  author = {Siyu Li and Kailun Yang and Hao Shi and Jiaming Zhang and Jiacheng Lin and Zhifeng Teng and Zhiyong Li},
  title = {{Bi-Mapper: Holistic BEV Semantic Mapping for Autonomous Driving}},
  journal = {IEEE Robotics Autom. Lett.},
  year = {2023},
  doi = {10.1109/lra.2023.3313932},
}
Figure