RESA: Recurrent Feature-Shift Aggregator for Lane Detection

Tu Zheng, Hao Fang, Yi Zhang, Wenjian Tang, Zheng Yang, Haifeng Liu, Deng Cai

Introduction

Lane detection is a fundamental task for autonomous driving systems, requiring the identification of "thin and long" structures that often occupy only a small fraction of the visual field. Traditional methods struggle with occlusions and poor visibility. The Recurrent Feature-Shift Aggregator (RESA) addresses these challenges by enabling efficient global spatial information aggregation.

The Challenge of Global Context

Standard Convolutional Neural Networks (CNNs) are limited by their local receptive fields. While they excel at detecting local features like edges or textures, they often fail to capture the global geometry of lanes, especially when markings are worn or obscured by traffic. Previous solutions like Spatial CNN (SCNN) utilized sequential message-passing, which, while effective at capturing context, is computationally expensive and slow for real-time deployment.

The RESA Architecture

The core innovation of this work is the Recurrent Feature-Shift Aggregator (RESA). Unlike sequential methods that pass information pixel-by-pixel, RESA slices the feature map and shifts these slices by varying strides.

RESA Overview: (a) Comparison between CNN and RESA results; (b) The Information Aggregation mechanism using recurrent feature shifts.

In each iteration $k$, slices are shifted with a stride $s_k = L/2^{K-k}$. This allows the network to aggregate information across a dimension of length $L$ in just $\log_2 L$ steps, transforming a linear bottleneck into a logarithmic one. This process is applied in four directions (top-to-bottom, bottom-to-top, left-to-right, and right-to-left) and is highly parallelizable on GPUs.

To complement RESA, the authors introduce the Bilateral Up-Sampling Decoder (BUSD). This decoder employs two branches: a Coarse-Grained Branch for rapid lane estimation via bilinear interpolation, and a Fine-Detailed Branch using transpose convolutions and non-bottleneck blocks to preserve intricate edge details.

Key Insights and Results

RESA allows the network to "conjecture" lane positions based on global context, such as the vanishing point or the position of adjacent lanes, even when a specific lane is entirely occluded. This modular enhancement can be dropped into existing architectures like ResNet or VGG as a spatial enrichment layer.

The model was evaluated on the CULane and TuSimple datasets:

  • Performance: On CULane, RESA achieved a state-of-the-art F1-measure of 75.3, significantly higher than SCNN's 71.6. On TuSimple, it reached 96.82% accuracy.
  • Efficiency: The RESA module is approximately 10 times faster than the SCNN message-passing scheme. The full RESA-50 model operates at roughly 28ms per frame (~36 FPS) on an NVIDIA 2080Ti, meeting the requirements for real-time autonomous driving.

Conclusion

RESA provides a modular, "plug-and-play" enhancement for lane detection that balances high accuracy with real-time performance. By replacing linear message passing with a logarithmic shift-and-add mechanism, it effectively captures the global spatial relationships necessary for robust lane detection in complex real-world scenarios. Future work may extend this mechanism to more complex road topologies like multi-lane intersections and forks.

Summary updated

Abstract

Lane detection is one of the most important tasks in self-driving. Due to various complex scenarios (e.g., severe occlusion, ambiguous lanes, etc.) and the sparse supervisory signals inherent in lane annotations, lane detection task is still challenging. Thus, it is difficult for the ordinary convolutional neural network (CNN) to train in general scenes to catch subtle lane feature from the raw image. In this paper, we present a novel module named REcurrent Feature-Shift Aggregator (RESA) to enrich lane feature after preliminary feature extraction with an ordinary CNN. RESA takes advantage of strong shape priors of lanes and captures spatial relationships of pixels across rows and columns. It shifts sliced feature map recurrently in vertical and horizontal directions and enables each pixel to gather global information. RESA can conjecture lanes accurately in challenging scenarios with weak appearance clues by aggregating sliced feature map. Moreover, we propose a Bilateral Up-Sampling Decoder that combines coarse-grained and fine-detailed features in the up-sampling stage. It can recover the low-resolution feature map into pixel-wise prediction meticulously. Our method achieves state-of-the-art results on two popular lane detection benchmarks (CULane and Tusimple). Code has been made available at: https://github.com/ZJULearning/resa.
BibTeX
@inproceedings{zheng2021resa,
  author = {Tu Zheng and Hao Fang and Yi Zhang and Wenjian Tang and Zheng Yang and Haifeng Liu and Deng Cai},
  title = {{RESA: Recurrent Feature-Shift Aggregator for Lane Detection}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2021},
  doi = {10.1609/aaai.v35i4.16469},
}
Figure