Learning To Detect 3D Lanes by Shape Matching and Embedding

Ruixin Liu, Zhihao Guan, Zejian Yuan, Ao Liu, Tong Zhou, Tang Kun, Erlong Li, Chao Zheng, Shuqi MeiWACV 2023

TL;DR

This paper introduces DSANet, a novel dual-level shape attention network for precise 3D lane detection from LiDAR point clouds. By jointly learning local segment shapes and global lane instance shapes with shape matching and embedding constraints, DSANet achieves state-of-the-art performance, particularly under stringent accuracy requirements.

What Problem Does This Paper Address?

Autonomous driving systems rely heavily on accurate 3D lane detection to understand the road ahead, enabling features like lane keeping and departure warnings. While RGB cameras have been used, their sensitivity to lighting and distortion limits precision. LiDAR point clouds offer rich, accurate geometric information, but detecting slender, complex lane structures in sparse, irregular point clouds remains challenging. Existing deep learning methods often struggle with precise localization, instance differentiation, and handling complex lane topologies, especially when requiring centimeter-level accuracy.

Methods

The core of this work is the Dual-Level Shape Attention Network (DSANet), which employs a two-branch architecture.

  • Shared Backbone: A combination of CNNs and self-attention modules extracts rich spatial features from the input pseudo-BEV image derived from LiDAR point clouds. This captures both local details and global context.
  • Segment Shape Branch: This branch focuses on local lane segments. It predicts:
  • Segment Existence Confidences: Whether a lane segment exists in a given grid.
  • Fine-grained Segment Shapes: Approximated as straight line segments with parameters like position, length, and orientation within the grid.
  • Shape Embeddings: Low-dimensional features encoding the approximate shape of the lane instance a segment belongs to.
  • Instance Shape Branch: This branch predicts global lane instance shapes, approximated as straight line segments connecting the start and end points of a lane. It outputs:
  • Instance Existence Confidences: Whether a lane instance exists.
  • Global Instance Shapes: Parameters defining the overall lane instance.

A key innovation is the introduction of shape matching and embedding loss functions. These losses jointly optimize the parameters of both branches, ensuring consistency between local segment shapes and global instance shapes. Specifically:

  • Segment Shape Matching Loss: Enforces constraints on segment existence, location, and geometry, using a 2D Gaussian distribution to represent lane segments for supervision.
  • Shape Embedding Loss: Constrains the lane instance shapes predicted from local feature maps.
  • Instance Shape Matching Loss: Supervises the global instance shapes and their existence confidences.

Finally, a Shape-Guided Segments Aggregator is proposed. This module uses the predicted global instance shapes as explicit clustering centers to group the detected local lane segments into complete lane instances. This approach aims to avoid issues with random initialization common in other clustering methods and ensures a more accurate number of detected lanes.

Datasets

The authors introduce and use the BEV-3DLanes dataset, a large-scale, real-world 3D lane dataset collected from LiDAR sensors across multiple cities. It contains 200 km of road data with high-density point clouds and centimeter-level ground truth annotations in a local coordinate system. Adjacent frames are spliced to capture longer lane structures. The dataset is processed into a pseudo-BEV image format for network input. The dataset consists of 29,874 training frames and 5,403 testing frames.

Key Insights and Ideas

  • Dual-Level Representation is Crucial: The paper highlights that effectively detecting 3D lanes requires considering both the fine-grained local segment shapes and the overarching global instance shapes.
  • Shape Matching for Precision: Jointly optimizing local segment shapes and global instance shapes through shape matching losses significantly improves precision, especially under tight accuracy thresholds. This enforces consistency between different levels of representation.
  • Shape Embeddings as Instance Signatures: Using shape embeddings to encode approximate instance shapes provides a more physically meaningful representation for grouping segments compared to generic embedding features.
  • Shape-Guided Aggregation for Instance Grouping: Using predicted instance shapes as explicit clustering centers for segments is more robust than traditional mean-shift clustering, leading to more accurate lane instance counts and preventing mis-grouping of close lanes.

Key Results

DSANet achieves state-of-the-art performance on the BEV-3DLanes dataset, outperforming several strong baselines (CondLaneNet, PINet, PolyLaneNet, LSTR, Lane-ATT) by a significant margin, particularly at stricter thresholds (5 cm, 10 cm).

  • 5 cm Threshold: DSANet achieves an F1 score of 72.79%, outperforming the next best method (CondLaneNet) by 3.3%.
  • 10 cm Threshold: DSANet achieves an F1 score of 86.26%, outperforming CondLaneNet by 0.8%.
  • 20 cm Threshold: DSANet achieves an F1 score of 91.59%, outperforming CondLaneNet by 0.58%.
  • Ablation Studies:
    • Segment shape matching constraints improve F1 score by 7.43% at 5 cm compared to using smooth-L1 loss on points.
  • The shape-guided segments aggregator improves F1 score by 4.24% at 5 cm compared to mean-shift.

Qualitatively, DSANet demonstrates superior performance in general cases, polylines, curves, and forks, producing complete and accurate lane results.

Limitations and Future Work

The authors acknowledge that the correlation between inter-frame results and the smoothness of predictions has not been fully studied. They suggest that incorporating multi-frame data for long-range 3D lane detection and tracking would be a valuable direction for future work.

Summary updated

BibTeX
@inproceedings{liu2023learning,
  author = {Ruixin Liu and Zhihao Guan and Zejian Yuan and Ao Liu and Tong Zhou and Tang Kun and Erlong Li and Chao Zheng and Shuqi Mei},
  title = {{Learning To Detect 3D Lanes by Shape Matching and Embedding}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2023},
}
Figure