Sparse Point Guided 3D Lane Detection

Chengtang Yao, Lidong Yu, Yuwei Wu, Yunde JiaICCV 2023 · OpenReview

Abstract

3D lane detection usually builds a dense correspondence between the front-view space and the BEV space to estimate lane points in the 3D space. 3D lanes only occupy a small ratio of the dense correspondence, while most correspondence belongs to the redundant background. This sparsity phenomenon bottlenecks valuable computation and raises the computation cost of building a high-resolution correspondence for accurate results. In this paper, we propose a sparse point-guided 3D lane detection, focusing on points related to 3D lanes. Our method runs in a coarse-to-fine manner, including coarse-level lane detection and iterative fine-level sparse point refinements. In coarse-level lane detection, we build a dense but efficient correspondence between the front view and BEV space at a very low resolution to compute coarse lanes. Then in fine-level sparse point refinement, we sample sparse points around coarse lanes to extract local features from the high-resolution front-view feature map. The high-resolution local information brought by sparse points refines 3D lanes in the BEV space hierarchically from low resolution to high resolution. The sparse point guides a more effective information flow and greatly promotes the SOTA result by 3 points on the overall F1-score and 6 points on several hard situations while reducing almost half memory cost and speeding up 2 times.

Introduction

3D lane detection is a cornerstone of modern autonomous driving, enabling critical features like lane centering and departure warnings. To function, these systems must translate 2D camera images (front-view) into a 3D "Bird’s Eye View" (BEV). Traditionally, researchers have relied on dense correspondence, attempting to map every pixel or feature from the front-view to a grid in the BEV.

However, lanes are inherently thin, sparse structures. In a typical road scene, the vast majority of a dense correspondence map represents redundant background, such as the sky or empty pavement. This redundancy creates a computational bottleneck: high-resolution mapping is necessary for precise localization but is prohibitively expensive when applied globally.

Comparison between dense and sparse correspondence. Dense mapping wastes computation on redundant background, whereas sparse mapping focuses on lane-related areas.

Methodology: Coarse-to-Fine Refinement

The proposed framework replaces wasteful dense mapping with a targeted sparse approach, operating in two primary stages.

1. Coarse-Level Detection

The model begins by building a very low-resolution, dense BEV map. This initial pass is computationally "cheap" and provides a rough estimate of lane locations. This coarse output serves as the foundation for subsequent refinement.

2. Iterative Fine-Level Sparse Point Refinement

Instead of upscaling the entire BEV map, the model samples sparse points only in the vicinity of the coarse lanes. These 3D points are projected back onto the high-resolution 2D front-view feature map to "fetch" local features from specific coordinates.

To ensure the lanes are both smooth and accurate, the model fuses three distinct data types:

  • Local Features: Fine-grained details retrieved via sparse points.
  • Global Features: A compressed vector representing the entire image context.
  • Point Coordinates: The spatial location of the sampled points.

A novel aggregation pipeline then uses a series of convolutions to propagate information along the lane, maintaining continuity and producing a final refined 3D lane prediction.

The overall pipeline of the model, illustrating the transition from coarse-level detection to iterative fine-level sparse point refinement.

Key Insights

  • The 2/5 Rule: Empirical observation showed that even within a small window near a lane, only about 40% of correspondences are beneficial; the rest are noise.
  • Sparsity as a Feature: By embracing the inherent sparsity of lanes, the model avoids "feature noise" from the background, increasing robustness in extreme weather or nighttime conditions.
  • Asymmetric Sampling: Ablation studies revealed that lateral (x-axis) precision is the primary driver for fetching valid features, suggesting that sampling more densely along this axis provides higher value than along the vertical (z-axis).

Experimental Results

The model was evaluated on the OpenLane and ONCE-3DLanes benchmarks, which include diverse environments and complex scenarios like uphill/downhill roads and extreme weather.

  • Accuracy: On OpenLane, the method improved the state-of-the-art (SOTA) F1-score by 3 points overall. In "hard" scenarios, the improvement reached up to 6 points.
  • Efficiency: The approach reduced memory consumption by nearly 50% and increased inference speed by 2x compared to the PersFormer baseline.
  • Qualitative Performance: Visualizations demonstrate superior maintenance of lane structure in rainy, cloudy, and nighttime scenes where dense methods often struggle with image noise.

Conclusion and Future Work

While the sparse point-guided approach significantly improves efficiency and accuracy, it remains dependent on the initial coarse detection pass. If the low-resolution pass misses a lane, the refinement stage cannot recover it. Future work may explore more adaptive refinement strategies or custom CUDA kernels to further optimize the sparse sampling process for production environments.

Summary updated

Abstract

3D lane detection usually builds a dense correspondence between the front-view space and the BEV space to estimate lane points in the 3D space. 3D lanes only occupy a small ratio of the dense correspondence, while most correspondence belongs to the redundant background. This sparsity phenomenon bottlenecks valuable computation and raises the computation cost of building a high-resolution correspondence for accurate results. In this paper, we propose a sparse point-guided 3D lane detection, focusing on points related to 3D lanes. Our method runs in a coarse-to-fine manner, including coarse-level lane detection and iterative fine-level sparse point refinements. In coarse-level lane detection, we build a dense but efficient correspondence between the front view and BEV space at a very low resolution to compute coarse lanes. Then in fine-level sparse point refinement, we sample sparse points around coarse lanes to extract local features from the high-resolution front-view feature map. The high-resolution local information brought by sparse points refines 3D lanes in the BEV space hierarchically from low resolution to high resolution. The sparse point guides a more effective information flow and greatly promotes the SOTA result by 3 points on the overall F1-score and 6 points on several hard situations while reducing almost half memory cost and speeding up 2 times.
BibTeX
@inproceedings{yao2023sparse,
  author = {Chengtang Yao and Lidong Yu and Yuwei Wu and Yunde Jia},
  title = {{Sparse Point Guided 3D Lane Detection}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2023},
  url = {https://openreview.net/forum?id=ROTQY5rKoz},
}
Figure