SuperFusion: Multilevel LiDAR-Camera Fusion for Long-Range HD Map Generation and Prediction

Hao Dong, Xianjing Zhang, Xuan Jiang, Jun Zhang, Jintao Xu, Rui Ai, Weihao Gu, Huimin Lu, Juho Kannala, Xieyuanli Chen

Introduction

High-definition (HD) maps are critical for autonomous vehicle navigation, providing essential semantic information such as lane boundaries and pedestrian crossings. However, most existing systems are limited to a short perception range of approximately 30 meters. This "short-sightedness" forces vehicles to make abrupt corrections at high speeds, leading to safety concerns and uncomfortable ride quality. SuperFusion addresses this by extending the HD map generation range to 90 meters—triple the industry standard—through a sophisticated multilevel sensor fusion framework.

Multilevel Fusion Architecture

SuperFusion leverages the complementary strengths of LiDAR and cameras. While LiDAR provides precise depth, its data becomes sparse at long ranges. Conversely, cameras offer rich semantic detail but lack inherent depth information. The framework integrates these sensors at three distinct levels:

  1. Data-Level Fusion: The system uses LiDAR points to provide dense depth supervision for camera images, improving the accuracy of the camera-to-Bird's-Eye View (BEV) transformation.
  2. Feature-Level Fusion: To combat LiDAR sparsity at 60-90m, a cross-attention mechanism allows image features to guide the prediction of missing LiDAR features.
  3. BEV-Level Fusion: A learned "flow field" aligns camera and LiDAR BEV features, correcting for calibration errors before final semantic prediction.
The SuperFusion architecture employs cross-attention to predict LiDAR BEV features from image data.

Experimental Results

Evaluated on the nuScenes dataset and a custom real-world dataset, SuperFusion significantly outperforms existing baselines like HDMapNet and BEVFusion. The most notable improvements occur in the long-range (60-90m) zone, where pedestrian crossing detection accuracy more than doubled compared to previous state-of-the-art methods.

SuperFusion provides significantly more stable and accurate long-range mapping compared to HDMapNet, enabling smoother path planning.

Beyond raw detection metrics, the impact on downstream tasks is substantial. By providing a longer look-ahead distance, SuperFusion increased the path planning success rate from under 50% to 72%, ensuring safer and more predictable vehicle trajectories.

Conclusion

SuperFusion demonstrates that multilevel fusion is key to overcoming the range limitations of current autonomous driving systems. By using cameras to "fill in" sparse LiDAR data and ensuring precise spatial alignment, the framework enables high-quality HD map generation at ranges previously considered unreachable. Future work will focus on optimizing computational efficiency for real-time deployment and improving robustness in adverse weather conditions.

Summary updated

Abstract

High-definition (HD) semantic map generation of the environment is an essential component of autonomous driving. Existing methods have achieved good performance in this task by fusing different sensor modalities, such as LiDAR and camera. However, current works are based on raw data or network feature-level fusion and only consider short-range HD map generation, limiting their deployment to realistic autonomous driving applications. In this paper, we focus on the task of building the HD maps in both short ranges, i.e., within 30 m, and also predicting long-range HD maps up to 90 m, which is required by downstream path planning and control tasks to improve the smoothness and safety of autonomous driving. To this end, we propose a novel network named SuperFusion, exploiting the fusion of LiDAR and camera data at multiple levels. We use LiDAR depth to improve image depth estimation and use image features to guide long-range LiDAR feature prediction. We benchmark our SuperFusion on the nuScenes dataset and a self-recorded dataset and show that it outperforms the state-of-the-art baseline methods with large margins on all intervals. Additionally, we apply the generated HD map to a downstream path planning task, demonstrating that the long-range HD maps predicted by our method can lead to better path planning for autonomous vehicles. Our code and self-recorded dataset will be available at https://github.com/haomo-ai/SuperFusion.
BibTeX
@inproceedings{dong2024superfusion,
  author = {Hao Dong and Xianjing Zhang and Xuan Jiang and Jun Zhang and Jintao Xu and Rui Ai and Weihao Gu and Huimin Lu and Juho Kannala and Xieyuanli Chen},
  title = {{SuperFusion: Multilevel LiDAR-Camera Fusion for Long-Range HD Map Generation and Prediction}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2024},
  doi = {10.1109/icra57147.2024.10611320},
}
Figure