MapFusion: A General Framework for 3D Object Detection with HDMaps

Jin Fang, Dingfu Zhou, Xibin Song, Liangjun Zhang

TL;DR

This paper introduces MapFusion, a novel framework that effectively integrates High Definition Maps (HDMaps) into existing 3D object detection pipelines for autonomous driving. By leveraging map information, MapFusion significantly reduces false positives and improves detection accuracy across various baseline detectors, demonstrating its broad applicability and effectiveness.

What Problem Does This Paper Address?

Autonomous driving systems rely heavily on accurate 3D object detection to perceive their surroundings. While LiDAR-based methods have advanced significantly, they still struggle with challenges like distinguishing between foreground and background objects and detecting sparse or distant objects. Existing fusion strategies often rely on cameras, which are susceptible to environmental conditions. This paper addresses the underutilization of High Definition Maps (HDMaps) – rich, centimeter-level road infrastructure data – as a complementary sensor to boost the robustness and accuracy of 3D object detection, particularly in mitigating false positives.

Methods

MapFusion is presented as a general, detector-independent framework. Its core components are:

  • FeatureAgg Module: This module is responsible for extracting features from HDMaps and fusing them with LiDAR point cloud features. The fusion is achieved through a simple concatenation of feature tensors, optionally followed by a 1x1 convolution for enhanced aggregation.
  • MapSeg Module: An auxiliary segmentation head designed to learn road structure information directly from the input point cloud. This module is trained using a binary cross-entropy loss against ground-truth map images and is only used during training to improve the feature extraction backbone.
  • 2D Feature Extractor: A lightweight convolutional neural network (CNN) that processes the rasterized HDMap data to extract high-level semantic features. This module is designed to maintain the spatial resolution of the input map.

The framework integrates these map-derived features with standard 3D object detection pipelines (like SECOND, PointPillars, and CenterPoint) by concatenating them with voxel features before the final detection head.

Datasets

The primary dataset used for evaluation is nuScenes. This is a large-scale multimodal dataset for autonomous driving, featuring 1,000 scenes with comprehensive annotations for 3D object detection. The authors specifically chose nuScenes because it provides access to HDMap data, which is crucial for their framework.

Key Insights and Ideas

  • HDMaps as a Powerful Prior: The paper strongly advocates for HDMaps not just as post-processing aids but as integral components within the detection network. The inherent geometric and semantic information in HDMaps provides a strong prior that can disambiguate challenging detection scenarios.
  • Detector Agnosticism is Key: By designing MapFusion as a modular framework, the authors demonstrate its ability to enhance a variety of existing 3D object detectors without requiring significant architectural changes to the baselines. This makes the approach highly practical for adoption.
  • Auxiliary Segmentation for Feature Enhancement: The MapSeg module, trained to segment road structures, acts as a valuable auxiliary task. This not only helps in learning better road representations but also implicitly improves the backbone's ability to extract relevant features for the primary detection task.
  • Simple Fusion Can Be Effective: The FeatureAgg module's straightforward concatenation approach, with a minor enhancement via 1x1 convolution, proves surprisingly effective, highlighting that complex fusion mechanisms aren't always necessary when dealing with complementary modalities.

Key Results

MapFusion consistently improved the performance of three baseline 3D object detectors (SECOND, PointPillars, and CenterPoint) on the nuScenes dataset.

  • NDS Score: Improvements ranged from 1.24% to 1.50%.
  • mAP (mean Average Precision): Improvements ranged from 1.27% to 2.79%.
  • Specific Classes: The framework showed notable gains for small objects like "Bicycle" (up to 2.07% improvement) and large objects like "Barrier" (up to 7.08% improvement). For "Traffic Cone," PointPillars saw a significant 7.39% AP improvement.
  • Qualitative Results: Visualizations (Figure 1) demonstrate a significant reduction in false positives when MapFusion is applied, particularly in scenarios where LiDAR alone struggles.

Limitations and Future Work

  • HDMap Availability: The paper acknowledges that HDMaps are not universally available. Future work includes developing map prediction methods and integrating more widely accessible regular maps.
  • Sensor Fusion Expansion: The current framework fuses LiDAR and HDMaps. The authors plan to extend MapFusion to incorporate other sensors like radar and cameras.
  • Computational Overhead: While the framework is designed to be lightweight, the addition of map processing and fusion does introduce some computational overhead, though it's presented as manageable.

Summary updated

Abstract

3D object detection is a key perception component in autonomous driving. Most recent approaches are based on LiDAR sensors only or fused with cameras. Maps (e.g., High Definition Maps), a basic infrastructure for intelligent vehicles, however, have not been well exploited for boosting object detection tasks. In this paper, we propose a simple but effective framework - MapFusion to integrate the map information into modern 3D object detector pipelines. In particular, we design a FeatureAgg module for HD Map feature extraction and fusion, and a MapSeg module as an auxiliary segmentation head for the detection backbone. Our proposed MapFusion is detector independent and can be easily integrated into different detectors. The experimental results of three different baselines on large public autonomous driving dataset demonstrate the superiority of the proposed framework. By fusing the map information, we can achieve 1.27 to 2.79 points improvements for mean Average Precision (mAP) on three strong 3D object detection baselines.
BibTeX
@inproceedings{fang2021mapfusion,
  author = {Jin Fang and Dingfu Zhou and Xibin Song and Liangjun Zhang},
  title = {{MapFusion: A General Framework for 3D Object Detection with HDMaps}},
  booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems},
  year = {2021},
  url = {https://openreview.net/forum?id=M-YMIPJsKy},
}
Figure