PriorLane: A Prior Knowledge Enhanced Lane Detection Approach Based on Transformer

Qibo Qiu, Haiming Gao, Wei Hua, Gang Huang, Xiaofei He

Abstract

Lane detection is a fundamental component of autonomous driving systems. This paper introduces PriorLane, a transformer-only framework that achieves state-of-the-art (SOTA) performance on CULane and TuSimple benchmarks. Beyond standard visual features, PriorLane incorporates low-cost local prior knowledge (such as road maps) to enhance segmentation performance. A novel Knowledge Embedding Alignment (KEA) module is utilized to align these knowledge embeddings with image features. Extensive experiments on the Zjlab dataset demonstrate that PriorLane outperforms existing SOTA methods by 2.82% mIoU when prior knowledge is integrated.

Introduction

Lane detection remains a challenging "thin and long" problem in computer vision. Traditional Convolutional Neural Networks (CNNs) often struggle to capture the long-range dependencies necessary to trace lanes that disappear into the horizon or are obscured by obstacles. While existing methods attempt to use "shape priors," they frequently fail under severe occlusions, visual noise (like rain or glare), and complex geometries like intersections.

PriorLane addresses these issues by leveraging "prior knowledge"—low-cost map data that provides a blueprint of where the road should be, regardless of current visual conditions. By treating this map data as explicit prompts within a transformer architecture, the model can effectively "fill in the blanks" when visual cues are missing.

Methodology

The PriorLane architecture featuring the MiT backbone, KEA module, and Fusion Encoder.

The PriorLane architecture consists of three primary components:

  1. Hierarchical Backbone (MiT): The model employs a Mixed Transformer (MiT-B5) as an encoder. MiT uses overlapped patch merging to maintain local structural integrity while capturing the global context essential for segmenting thin lane lines.
  2. Knowledge Embedding Alignment (KEA): Prior knowledge, represented as Bird’s Eye View (BEV) grid maps, is often misaligned with the camera perspective due to coarse GPS data. The KEA module utilizes a Spatial Transformer Network (STN) and Active Rotation Filters (ARFs) to rotate and shift the map data, ensuring precise alignment with image features.
  3. Fusion Transformer (FT): Instead of simple concatenation, the model treats map data as explicit prompts. A dedicated Fusion Transformer uses self-attention to merge image features with knowledge embeddings, allowing the model to prioritize map data in regions where visual confidence is low.

Experimental Results

The researchers evaluated PriorLane on CULane, TuSimple, and a new custom dataset, Zjlab. The Zjlab dataset includes 869 frames with corresponding BEV grid maps to facilitate the training of the PriorLane framework.

Quantitative Performance

On the CULane dataset, the transformer-only baseline (MiT-Lane) achieved a 76.27 F1-measure, significantly outperforming established CNN-based models like RESA (74.04) and SCNN (73.58). On the Zjlab dataset, the full PriorLane-KEA model achieved 73.78% mIoU, a 2.82% improvement over the SOTA without prior knowledge.

Qualitative Robustness

Qualitative analysis reveals that PriorLane excels in scenarios where traditional models fail. It accurately predicts dotted lines and lanes hidden behind vehicles, and it can effectively distinguish between standard lane markings and stop lines at complex intersections.

Qualitative comparison on the Zjlab dataset showing PriorLane-KEA's superior performance in occluded and complex scenes compared to SOTA methods.

Conclusion and Future Work

PriorLane demonstrates that pure transformer architectures, when augmented with aligned prior knowledge, provide a robust solution for lane detection. While the system currently requires meter-level positioning, future work will focus on handling scenarios with zero GPS signal and expanding the types of prior knowledge used (e.g., traffic signs and elevation maps) to further harden the system against extreme environmental conditions.

Summary updated

Abstract

Lane detection is one of the fundamental modules in self-driving. In this paper we employ a transformer-only method for lane detection, thus it could benefit from the blooming development of fully vision transformer and achieve the state-of-the-art (SOTA) performance on both CULane and TuSimple benchmarks, by fine-tuning the weight fully pre-trained on large datasets. More importantly, this paper proposes a novel and general framework called PriorLane, which is used to enhance the segmentation performance of the fully vision transformer by introducing the low-cost local prior knowledge. Specifically, PriorLane utilizes an encoder-only transformer to fuse the feature extracted by a pre-trained segmentation model with prior knowledge embeddings. Note that a Knowledge Embedding Alignment (KEA) module is adapted to enhance the fusion performance by aligning the knowledge embedding. Extensive experiments on our Zjlab dataset show that PriorLane outperforms SOTA lane detection methods by a 2.82% mIoU when prior knowledge is employed, and the code will be released at: https://github.com/vincentqqb/PriorLane.
BibTeX
@inproceedings{qiu2023priorlane,
  author = {Qibo Qiu and Haiming Gao and Wei Hua and Gang Huang and Xiaofei He},
  title = {{PriorLane: A Prior Knowledge Enhanced Lane Detection Approach Based on Transformer}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2023},
  doi = {10.1109/icra48891.2023.10161356},
}
Figure