PriorLane: A Prior Knowledge Enhanced Lane Detection Approach Based on Transformer

Qibo Qiu, Haiming Gao, Wei Hua, Gang Huang, Xiaofei He

TL;DR

PriorLane is a transformer-only lane detection framework that significantly boosts performance by integrating low-cost "prior knowledge" (like road maps) as explicit prompts. By using a hierarchical vision transformer and a novel alignment module, it achieves state-of-the-art results on major benchmarks and excels in challenging scenarios like heavy occlusions and blurry distant markings.

What Problem Does This Paper Address?

Lane detection is a cornerstone of autonomous driving, but it remains a "thin and long" problem. Traditional Convolutional Neural Networks (CNNs) often struggle to capture the long-range dependencies required to trace a lane that disappears into the horizon or is blocked by a truck.

While existing methods use "shape priors" (assuming lanes are generally straight or curved), they often fail when visual cues are missing due to:

  1. Severe Occlusions: Vehicles or pedestrians blocking the view.
  2. Visual Noise: Rain, glare, or "fuzzy" markings in the distance.
  3. Complex Geometry: Intersections and forks where standard lane shapes break down.

The authors argue that we shouldn't rely solely on the pixels in the current frame. Instead, we should leverage "prior knowledge"—low-cost map data that tells the model where the road should be.

Methods

The authors propose a three-part architecture:

  1. The Backbone (MiT): They employ a Mixed Transformer (MiT-B5) as a hierarchical encoder. Unlike standard Transformers, MiT uses overlapped patch merging to maintain local structure while capturing global context, making it ideal for segmenting thin lane lines.
  2. Knowledge Embedding Alignment (KEA): This is the core innovation. Prior knowledge (represented as a Bird’s Eye View (BEV) grid map of road freespace) is often misaligned with the camera's perspective due to coarse GPS/positioning. The KEA module uses a Spatial Transformer Network (STN) and Active Rotation Filters (ARFs) to rotate and shift the map data so it aligns perfectly with the image features.
  3. Fusion Transformer (FT): Instead of simply "stacking" the map and the image, the model treats the map data as explicit prompts. A dedicated Fusion Transformer uses self-attention to merge the image features with these map-based "knowledge embeddings," allowing the model to "fill in the blanks" where lanes are occluded.

Datasets

The researchers evaluated their approach on three benchmarks:

  • CULane: A massive, widely-used dataset for urban and highway lane detection.
  • TuSimple: A standard benchmark focused on highway scenarios.
  • Zjlab (New): A custom dataset introduced by the authors containing 869 frames (and 10,000 visualization images) collected at Zhejiang Lab. Crucially, this dataset includes BEV grid maps to provide the necessary prior knowledge for training the PriorLane framework.

Key Insights and Ideas

  • Transformers > CNNs for Lanes: The paper demonstrates that a "pure" transformer (MiT-Lane) outperforms established CNN-based models like SCNN and RESA. This is because the self-attention mechanism naturally "links" distant parts of a lane line better than local convolutional kernels.
  • Map Data as a "Prompt": By treating map data as a prompt rather than just another input channel, the model learns a more sophisticated relationship between "where the road is" and "what the camera sees."
  • Alignment is Everything: The authors found that simply adding map data isn't enough. The KEA module’s ability to handle orientation-invariant features (via ARFs) was critical for making the prior knowledge useful in real-world, "noisy" positioning scenarios.

Key Results

  • State-of-the-Art Performance: On the CULane dataset, the transformer-only baseline (MiT-Lane) achieved a 76.27 F1-measure, outperforming RESA (74.04) and SCNN (73.58).
  • The Power of Prior Knowledge: On the Zjlab dataset, the full PriorLane-KEA model achieved 73.78% mIoU, which is a 2.82% improvement over the SOTA without prior knowledge.
  • Robustness: Qualitative results show that PriorLane can accurately predict "dotted" lines and lanes hidden behind obstacles where other models see only empty pavement. It can also distinguish between standard lane markings and stop lines at intersections.

Limitations and Future Work

  • Positioning Costs: While the authors use "low-cost" maps, the system still requires meter-level positioning. In environments with zero GPS signal (like some tunnels or deep urban canyons), the prior knowledge might become a liability if not handled carefully.
  • Perception Range Trade-off: The authors noted a "peak" in performance at a 20-meter perception range. If the range is too large, the resolution of the distant map data becomes too low for the attention mechanism to process effectively.
  • Future Directions: The authors suggest expanding the dataset to include more types of prior knowledge (e.g., traffic signs, elevation maps) to further harden the segmentation against extreme weather and lighting.

Summary updated

Abstract

Lane detection is one of the fundamental modules in self-driving. In this paper we employ a transformer-only method for lane detection, thus it could benefit from the blooming development of fully vision transformer and achieve the state-of-the-art (SOTA) performance on both CULane and TuSimple benchmarks, by fine-tuning the weight fully pre-trained on large datasets. More importantly, this paper proposes a novel and general framework called PriorLane, which is used to enhance the segmentation performance of the fully vision transformer by introducing the low-cost local prior knowledge. Specifically, PriorLane utilizes an encoder-only transformer to fuse the feature extracted by a pre-trained segmentation model with prior knowledge embeddings. Note that a Knowledge Embedding Alignment (KEA) module is adapted to enhance the fusion performance by aligning the knowledge embedding. Extensive experiments on our Zjlab dataset show that PriorLane outperforms SOTA lane detection methods by a 2.82% mIoU when prior knowledge is employed, and the code will be released at: https://github.com/vincentqqb/PriorLane.
BibTeX
@inproceedings{qiu2023priorlane,
  author = {Qibo Qiu and Haiming Gao and Wei Hua and Gang Huang and Xiaofei He},
  title = {{PriorLane: A Prior Knowledge Enhanced Lane Detection Approach Based on Transformer}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2023},
  doi = {10.1109/icra48891.2023.10161356},
}

Loading related content...

Figure