TL;DR
This paper introduces HeightLane, a novel monocular 3D lane detection method that tackles the challenge of varying road slopes by explicitly predicting a dense heightmap of the ground. This heightmap guides a deformable attention-based spatial transform to accurately project 2D image features into a 3D Bird's-Eye-View (BEV) representation, achieving state-of-the-art performance on the OpenLane dataset.
What Problem Does This Paper Address?
Accurately detecting 3D lane markings from a single camera image is a crucial but challenging task for autonomous driving. The primary difficulty lies in inferring depth information from a 2D image, especially on uneven road surfaces with varying slopes and elevations. Previous methods often assumed a flat ground or a simplified 2-degree-of-freedom (2-DoF) ground model, which leads to misalignments between 2D image features and their corresponding 3D representations in BEV, particularly in complex real-world scenarios. This paper aims to overcome these limitations by providing a more accurate and detailed representation of the ground.
Methods
HeightLane introduces a new approach centered around a heightmap.
- Heightmap Prediction: Instead of relying on simplified ground models, HeightLane first defines a dense BEV grid and then predicts a heightmap for this grid. This is achieved by using multi-slope heightmap anchors that are projected onto the image to sample front-view features. These sampled features are then used to predict the heightmap.
- Height-Guided Spatial Transformation: The predicted heightmap is then used to guide a deformable attention-based spatial feature transform. This framework uses the heightmap to accurately align 2D front-view features with the BEV grid. BEV queries interact with front-view features (acting as keys and values) through deformable attention, where the heightmap helps determine precise reference points in the image for each BEV query. This explicitly transforms image features into a spatially accurate BEV representation.
- BEV Representation: The transformed BEV features are then fed into a lane detection head that uses a keypoint-based representation to output the final 3D lane detection.
Datasets
- OpenLane Dataset [2]: Used for both training and validation. This dataset is built upon the Waymo dataset and provides a variety of road conditions, weather, and lighting scenarios.
- Waymo Dataset [21]: LiDAR data from Waymo was used to generate ground truth heightmaps for the OpenLane dataset, as OpenLane itself does not directly provide this information.
Key Insights and Ideas
- Explicit Ground Height Modeling is Crucial: The paper strongly advocates for explicitly modeling the ground's height, rather than relying on simplified planar assumptions. This detailed ground representation is key to overcoming the challenges of varying road slopes.
- Heightmap as a Bridge: The heightmap acts as a critical intermediary, enabling accurate spatial transformation from the front-view image domain to the BEV domain. It effectively bridges the gap between 2D image information and the 3D spatial understanding required for lane detection.
- Deformable Attention Leverages Height: By incorporating the heightmap into the deformable attention mechanism, the model can precisely determine where to sample features from the front-view image, leading to more accurate BEV feature representation.
Key Results
- State-of-the-Art Performance: HeightLane achieved the highest F-score (62.7%) on the OpenLane validation set, outperforming existing state-of-the-art models like LATR.
- Robustness in Complex Scenarios: The method demonstrated strong performance in challenging scenarios such as curves, intersections, and areas with changing slopes, accurately detecting lanes even when interrupted or occluded.
- Competitive X-Error: HeightLane achieved the second-best X-error, indicating accurate lateral positioning of detected lanes.
- Ablation Studies:
- Multi-slope heightmap anchors significantly improved performance over single-slope anchors.
- Using 0°, ±3°, and ±5° slopes for anchors yielded the best results, balancing performance and computational cost.
- When using ground truth heightmaps (Ours (GT)), the model's spatial transform framework achieved competitive results compared to multi-modal (LiDAR + camera) methods, highlighting the effectiveness of the transformation itself.
Limitations and Future Work
- Ground Truth Heightmap Generation: The paper acknowledges that generating ground truth heightmaps requires LiDAR data, which is not always available. The method relies on the Waymo dataset for this purpose.
- Computational Cost: While the paper balances performance and efficiency, increasing the number of heightmap anchors can increase computational cost.
- Future Work: The paper's success in leveraging height information for accurate spatial transformation suggests that this approach could be extended to other 3D perception tasks beyond lane detection. Further exploration into more efficient heightmap prediction and integration with other sensor modalities could also be promising.
Summary updated