Robust Lane Detection via Expanded Self Attention

Minhyeok Lee, Junhyeop Lee, Dogyoon Lee, Woo Jin Kim, Sang-Won Hwang, Sangyoun LeeWACV 2021 · OpenReview

TL;DR

This paper introduces the Expanded Self Attention (ESA) module, a novel self-attention mechanism designed to improve lane detection robustness in challenging scenarios like occlusions and extreme lighting. By predicting lane confidence along horizontal and vertical directions, ESA effectively extracts global contextual information without increasing inference time, achieving state-of-the-art results on CULane and BDD100K datasets.

What Problem Does This Paper Address?

Autonomous vehicles rely heavily on accurate lane detection to navigate safely. While deep learning has significantly advanced this field, current methods struggle in difficult conditions such as heavy traffic congestion, poor lighting (nighttime, shadows), and situations where lanes are partially or fully obscured by other vehicles or road features. Humans can infer lane positions even with limited visual cues by understanding the broader scene context. This paper aims to bridge this gap by enabling deep learning models to better capture and utilize global contextual information, specifically to infer occluded lane locations.

Methods

The core contribution is the Expanded Self Attention (ESA) module. This module is designed to be easily integrated into existing encoder-decoder-based lane detection architectures without adding computational overhead during inference. The ESA module works by predicting the "confidence" of a lane along specific directions.

Key aspects of the ESA module:

  • Inspired by Lane Geometry: Lanes are essentially linear structures. The ESA module leverages this by predicting confidence along the vertical (HESA - Horizontal Expanded Self Attention) and horizontal (VESA - Vertical Expanded Self Attention) axes.
  • Occlusion Prediction: By predicting confidence, ESA helps the model understand where lane information is weak or missing due to occlusions. This allows the model to infer these occluded locations by leveraging global context.
  • No Additional Supervision: A significant advantage is that ESA does not require extra annotations for occlusions. It learns to predict confidence implicitly.
  • Inference-Time Removal: The ESA module is only active during training. During inference, it's removed, meaning the model's speed remains the same as the original baseline.
  • HESA and VESA: These two variants differ in the direction of expansion of the ESA encoder's output. HESA expands horizontally, while VESA expands vertically. The paper suggests using both (H&VESA) often yields the best results.

Datasets

The proposed method was evaluated on three popular lane detection benchmarks:

  • TuSimple: Primarily highway images with consistent lighting and less congestion.
  • CULane: A challenging dataset featuring crowded urban environments with varying lighting and significant occlusions.
  • BDD100K: A large-scale dataset with diverse driving conditions, including various lighting and weather, and a higher number of lanes per image.

Key Insights and Ideas

  • Global Context is King for Occlusions: The paper strongly emphasizes that for lane detection, especially in occluded scenarios, understanding the broader scene context is more critical than precisely identifying the shape of the occlusion itself.
  • Confidence as a Proxy for Occlusion: Predicting lane confidence (how sure the model is about a lane's presence at a given pixel) is a clever way to implicitly learn about occlusions without explicit labels.
  • Simplicity and Efficiency: The ESA module's design is simple, modular, and crucially, doesn't increase inference time. This makes it highly practical for real-world autonomous driving systems where speed is paramount.
  • Complementary Nature of HESA and VESA: The study found that combining both HESA and VESA modules often leads to superior performance, suggesting that capturing contextual information from both horizontal and vertical perspectives is beneficial, especially in complex datasets like BDD100K.

Key Results

  • State-of-the-Art Performance: The proposed ERFNet-H&VESA model achieved state-of-the-art results on the CULane (F1-score of 74.2) and BDD100K (Accuracy IoU of 60.24%) datasets.
  • Robustness: The method demonstrated significant improvements in challenging low-light and occlusion scenarios, as evidenced by qualitative results and performance gains on CULane.
  • Efficiency: The ESA module does not increase inference time or the number of parameters compared to the baseline model.
  • TuSimple Performance: While showing improvement on TuSimple, the method was less dominant there, likely because the dataset has fewer challenging occlusion and lighting conditions where ESA's strengths lie.

Limitations and Future Work

  • Dataset Specificity of $\gamma$: The optimal value for the regularization parameter $\gamma$ in the ESA loss function was found to be dataset-dependent (0.8 for CULane/BDD100K, 0.9 for TuSimple). This suggests that fine-tuning this parameter might be necessary for different driving environments.
  • TuSimple's Less Challenging Nature: The paper acknowledges that ESA's benefits are less pronounced on datasets like TuSimple, which lack significant occlusions and extreme lighting. This implies that ESA is particularly suited for more complex, real-world driving scenarios.
  • Future Work: The paper doesn't explicitly detail future work, but the success of ESA suggests potential applications in other computer vision tasks that require robust contextual understanding in the presence of missing or degraded information. Further exploration into optimizing the $\gamma$ parameter or developing adaptive ESA configurations could also be future directions.

Summary updated

Abstract

The image-based lane detection algorithm is one of the key technologies in autonomous vehicles. Modern deep learning methods achieve high performance in lane detection, but it is still difficult to accurately detect lanes in challenging situations such as congested roads and extreme lighting conditions. To be robust on these challenging situations, it is important to extract global contextual information even from limited visual cues. In this paper, we propose a simple but powerful self-attention mechanism optimized for lane detection called the Expanded Self Attention (ESA) module. Inspired by the simple geometric structure of lanes, the proposed method predicts the confidence of a lane along the vertical and horizontal directions in an image. The prediction of the confidence enables estimating occluded locations by extracting global contextual information. ESA module can be easily implemented and applied to any encoder-decoder-based model without increasing the inference time. The performance of our method is evaluated on three popular lane detection benchmarks (TuSimple, CULane and BDD100K). We achieve state-of-the-art performance in CULane and BDD100K and distinct improvement on TuSimple dataset. The experimental results show that our approach is robust to occlusion and extreme lighting conditions.
BibTeX
@inproceedings{lee2021robust,
  author = {Minhyeok Lee and Junhyeop Lee and Dogyoon Lee and Woo Jin Kim and Sang-Won Hwang and Sangyoun Lee},
  title = {{Robust Lane Detection via Expanded Self Attention}},
  booktitle = {IEEE/CVF Winter Conference on Applications of Computer Vision},
  year = {2021},
  url = {https://openreview.net/forum?id=R3MRFppbW_},
}
Figure