FastDraw: Addressing the Long Tail of Lane Detection by Adapting a Sequential Prediction Network

Jonah Philion

Abstract

FastDraw is a high-speed lane detection framework that replaces traditional, brittle post-processing with a learned sequential decoder integrated directly into a convolutional neural network. By leveraging unsupervised style transfer, the model can adapt to difficult "long tail" driving conditions—such as rain or night—without requiring any additional manual annotations.

Introduction

Lane detection is a cornerstone of autonomous driving, yet most existing models follow a rigid three-step template: predict pixel-wise lane probability, threshold those pixels, and cluster them into individual lanes using algorithms like RANSAC. This approach suffers from two major flaws. First, the clustering step is non-differentiable and separate from the neural network, preventing the model from learning optimal grouping strategies. Second, models trained on clean, sunny datasets fail in the "long tail" of real-world conditions—rain, snow, glare, or night—where visual textures differ significantly from training data.

Methods

The authors introduce FastDraw, which treats lane detection as a sequential prediction problem handled entirely within a CNN architecture.

The FastDraw Architecture

Instead of simple segmentation, FastDraw utilizes three specialized "heads" within its decoder. One head identifies if a pixel belongs to a lane, while the other two predict a categorical distribution over the pixels in the rows immediately above and below. This allows the network to learn local continuity: "If I am at this pixel, the next part of the lane is likely $L$ pixels to the left or right in the next row."

The FastDraw architecture featuring a CNN encoder and a sequential decoder with multiple prediction heads.

Autoregressive Sequential Decoding

While the logic is sequential, the implementation is fully convolutional, enabling the representation of an arbitrary number of lanes of any length while maintaining high inference speeds. To mitigate "drift" common in sequential models, the authors employ a denoising technique during training, adding Gaussian noise to ground-truth starting points to force the network to learn error correction.

Unsupervised Domain Adaptation

To handle new environments, FastDraw employs MUNIT (Multimodal Unsupervised Image-to-Image Translation). Labeled sunny images are translated into the styles of rainy or nighttime scenes. Because style transfer preserves geometry, the original lane labels remain valid, creating "pseudo-labeled" training data for difficult conditions without manual effort.

Results

FastDraw achieves a high inference speed of 90 FPS on a GTX 1080, significantly outperforming sequential competitors like PolyLine-RNN (5.7 FPS). On the TuSimple benchmark, it reached 95.2% accuracy, remaining competitive with state-of-the-art models while requiring less post-processing.

Comparison of lane detection performance on out-of-domain images before and after adaptation.

On the CULane dataset, FastDraw outperformed the ResNet-50 baseline specifically on curves (65.2 vs 59.8 IoU), validating its polyline representation. Qualitative results demonstrate that the adapted model successfully detects lanes in heavy rain and night scenes where the base model, trained only on TuSimple, failed completely.

Conclusion

FastDraw provides a robust, high-speed solution for lane detection that bridges the gap between segmentation and geometric modeling. By integrating sequential logic into a differentiable framework and utilizing unsupervised adaptation, it effectively addresses the "long tail" of driving conditions that challenge traditional autonomous systems.

Summary updated

Abstract

The search for predictive models that generalize to the long tail of sensor inputs is the central difficulty when developing data-driven models for autonomous vehicles. In this paper, we use lane detection to study modeling and training techniques that yield better performance on real world test drives. On the modeling side, we introduce a novel fully convolutional model of lane detection that learns to decode lane structures instead of delegating structure inference to post-processing. In contrast to previous works, our convolutional decoder is able to represent an arbitrary number of lanes per image, preserves the polyline representation of lanes without reducing lanes to polynomials, and draws lanes iteratively without requiring the computational and temporal complexity of recurrent neural networks. Because our model includes an estimate of the joint distribution of neighboring pixels belonging to the same lane, our formulation includes a natural and computationally cheap definition of uncertainty. On the training side, we demonstrate a simple yet effective approach to adapt the model to new environments using unsupervised style transfer. By training FastDraw to make predictions of lane structure that are invariant to low-level stylistic differences between images, we achieve strong performance at test time in weather and lighting conditions that deviate substantially from those of the annotated datasets that are publicly available. We quantitatively evaluate our approach on the CVPR 2017 Tusimple lane marking challenge, difficult CULane datasets [8], and a small labeled dataset of our own and achieve competitive accuracy while running at 90 FPS.
BibTeX
@inproceedings{philion2019fastdraw,
  author = {Jonah Philion},
  title = {{FastDraw: Addressing the Long Tail of Lane Detection by Adapting a Sequential Prediction Network}},
  booktitle = {Workshops @ CVPR},
  year = {2019},
}
Figure