LHFFNet: A hybrid feature fusion method for lane detection

Youchen Kao, Shengbing Che, Sha Zhou, Shen‐Yi Guo, Xu Zhang, Wanqin WangScientific 2024 · DOI

TL;DR

LHFFNet is a novel lane detection framework that uses a hybrid feature fusion approach to tackle the "far-end" problem—where distant lane lines become too narrow and visually similar to be accurately detected. By integrating multi-head self-attention, spatially separable convolutions, and strip pooling, the model achieves state-of-the-art performance on major benchmarks like TuSimple (96.84% accuracy) and CULane (75.9% F1 score).

What Problem Does This Paper Address?

Lane detection is a cornerstone of autonomous driving, yet it remains surprisingly difficult. The authors identify three primary hurdles:

  1. Scale Variation: Due to perspective, lane lines are wide in the foreground but taper into tiny, narrow strips at the horizon (the "far-end").
  2. Visual Noise: Complex scenes involving shadows, glare, and vehicle occlusions make it hard to distinguish lanes from the road surface.
  3. Information Imbalance: In a typical road image, background pixels vastly outnumber lane pixels. Traditional pooling operations (which summarize image regions) often "wash out" these thin, elongated lane features with irrelevant background data.

Existing models like PINet often struggle with redundant background information and a limited receptive field, leading to "point accumulation" or deviations at the far end of the road.

Methods

LHFFNet builds upon the Stacked Hourglass Network (SHN) backbone and introduces three specialized modules to refine feature extraction:

  • Multi-space Attention Enhancement Module (MAEM): This module uses Multi-Head Self-Attention (MHSA) with relative position encoding. Unlike standard convolutions that only look at local neighbors, MAEM allows the network to capture global context, helping the model "understand" the continuity of a lane line across the entire image.
  • Space Concern Distinguish Module (SCDM): Standard "skip connections" (which pass data between layers) treat all pixels equally. SCDM introduces a spatially separable convolutional branch that assigns higher weights to key lane regions. This helps the model filter out background noise and focus on the actual lane structure during feature fusion.
  • Strip Pooling Module (SPM): Traditional pooling uses square kernels (e.g., 2x2). SPM uses long, narrow kernels (horizontal and vertical strips). This is a "form-fits-function" approach: since lane lines are inherently long and thin, strip pooling is far more effective at capturing their anisotropic (direction-dependent) context without including too much irrelevant background.

Datasets

The authors evaluated LHFFNet on two industry-standard benchmarks:

  • TuSimple: A large-scale dataset focused on highway scenes under stable lighting. It includes 3,268 training and 2,782 test images.
  • CULane: A significantly more challenging dataset containing 133,235 images across nine complex scenarios, including "Night," "Shadow," "No line," and "Crowded" urban environments.

Key Insights and Ideas

  • Attention as a Bridge: The use of MHSA isn't just a trend; it specifically solves the problem of correlating the wide, clear lane markings in the foreground with the faint, narrow markings at the horizon.
  • Anisotropic Pooling is Essential: The "aha moment" here is recognizing that square pooling kernels are fundamentally ill-suited for linear features. By using strip pooling, the model can "see" the entire length of a lane line without getting distracted by the cars or trees next to it.
  • Smart Skip Connections: By adding spatial weights to the jumping layers (SCDM), the authors proved that how you fuse features is just as important as what features you extract. This prevents deep semantic information from being blurred by shallow, noisy local features.

Key Results

LHFFNet demonstrated significant improvements over the baseline (PINet) and other mainstream models:

  • TuSimple: Reached 96.84% accuracy, outperforming models like SCNN and ENet-SAD. Notably, it reduced False Positives (FP) and False Negatives (FN) compared to the baseline.
  • CULane: Achieved a top F1 score of 75.9%. In specific difficult scenarios, it showed dramatic gains over PINet, such as a 4.4% improvement in "Curve" scenes and a 4.1% improvement in "Arrow" markings.
  • Qualitative Success: Visual results show that LHFFNet produces much smoother, more accurate lane fits at the far end of the road, even when vehicles partially occlude the lane.

Limitations and Future Work

While LHFFNet excels in complex lighting and geometry, the authors acknowledge a remaining "blind spot": extreme weather. The model's performance still needs enhancement for scenarios involving heavy rain or muddy roads where lane markings are physically obscured.

The next phase of their research involves constructing a dedicated dataset for rainy-day driving and further refining the model architecture to handle these low-visibility conditions.

Summary updated

Abstract

Lane line images have the essential attribute of large-scale variation and complex scene information, and the similarity between adjacent lane lines is high, which can easily cause classification errors. And remote lane lines are difficult to recognize due to visual angle changes in width. To address this issue, this paper proposes an effective lane detection framework, which is a hybrid feature fusion network that enhances multiple spatial features and distinguishes key features throughout the entire lane line segment. It enhances and fuses lane line features at multiscale to enhance the feature representation of lane line images, especially at the far end. Firstly, in order to enhance the correlation of multiscale lane features, a multi-head self attention is used to construct a multi-space attention enhancement module for feature enhancement in multispace. Secondly, a spatial separable convolutional branch is designed for the jumping layer structure connecting multiscale lane line features. While retaining feature information of different scales, important lane areas in multiscale feature information are emphasized through the allocation of spatial attention weights. Finally, considering that lane lines are elongated areas in the image, and the background information in the image is much more abundant than lane line information, the flexibility of traditional pooling operations in capturing widely existing anisotropic contexts in actual environments is limited. Therefore, before embedding feature output branches, strip pooling is introduced to refine the representation of lane line information and optimize model performance. The experimental results show that the accuracy on the TuSimple dataset reaches 96.84%, and the F1 score on the CULane dataset reaches 75.9%.
BibTeX
@inproceedings{kao2024lhffnet,
  author = {Youchen Kao and Shengbing Che and Sha Zhou and Shen‐Yi Guo and Xu Zhang and Wanqin Wang},
  title = {{LHFFNet: A hybrid feature fusion method for lane detection}},
  booktitle = {Scientific},
  year = {2024},
  doi = {10.1038/s41598-024-66913-1},
}
Figure