FastDraw: Addressing the Long Tail of Lane Detection by Adapting a Sequential Prediction Network.

Jonah Philion

Introduction

Lane detection is a foundational task for autonomous vehicles, yet it remains plagued by two primary challenges: the "long tail" of environmental conditions and "post-processing bloat." Most existing datasets are collected in fair weather, leading standard models to overfit to sunny highway textures. When these models encounter heavy rain, glare, or night-time conditions, performance often degrades significantly. Furthermore, traditional pipelines typically rely on semantic segmentation followed by non-differentiable clustering algorithms (like RANSAC). These heuristics are computationally expensive, difficult to tune, and prevent the model from learning the global structure of a lane during training.

FastDraw addresses these issues by replacing brittle clustering with a learned "drawing" mechanism. By combining a high-speed sequential prediction network with unsupervised style transfer, the framework achieves a blistering 90 FPS and successfully navigates challenging driving conditions without requiring additional manual labels.

Methods

The core of FastDraw is a Sequential Prediction Network that treats lane detection as a path-following problem rather than a simple pixel-classification task.

The "Drawing" Decoder

Instead of a standard heatmap output, the network utilizes a decoder with three distinct heads. One head performs binary segmentation to identify lane pixels. The other two heads predict categorical distributions for the pixels in the rows immediately above and below the current pixel. This allows the network to iteratively ask: "If this is a lane, where does it go next?"

The FastDraw architecture, featuring a convolutional backbone and a sequential prediction decoder with three output heads.

While the logic is sequential, the architecture remains fully convolutional, allowing it to leverage GPU acceleration. To prevent the model from "drifting" off the lane at test time, the authors employ a denoising technique during training, adding Gaussian noise to ground-truth coordinates and training the network to point back toward the true lane center.

Unsupervised Style Transfer (MUNIT)

To handle the long tail of sensor inputs, the authors use Multimodal Unsupervised Image-to-image Translation (MUNIT). This allows them to translate labeled images from a source domain (e.g., sunny California) into the "style" of a target environment (e.g., rainy Massachusetts). This process preserves the underlying geometry of the lanes while changing the visual texture, effectively generating "synthetic" training data for difficult conditions without the need for new manual annotations.

Key Results and Insights

FastDraw demonstrates that geometry is a more robust feature than texture. By using style transfer, the model learns that a lane is a structural curve, regardless of whether the asphalt is dry or reflecting streetlights in a storm.

  • Speed: FastDraw runs at 90.31 FPS on a GTX 1080, significantly outperforming models like H-Net (52.6 FPS) and PolyLine-RNN (5.7 FPS).
  • Accuracy: On the TuSimple benchmark, FastDraw achieved 95.2% accuracy.
  • Robustness: The "adapted" model showed a dramatically better precision/recall tradeoff on out-of-domain images. It successfully identified lanes in night and rain conditions where the base model, trained only on standard datasets, failed entirely.
Comparison of standard FastDraw versus Adapted FastDraw in out-of-domain conditions (night and rain).

Additionally, because the model predicts a categorical distribution over the next pixel's location, the standard deviation of that distribution serves as a built-in uncertainty metric. If the prediction is highly spread out, the system can signal that it is "confused" due to occlusions or poor visibility, providing a safety-critical signal for the vehicle's planning system.

Conclusion

By moving clustering logic into the neural network's weights and utilizing unsupervised adaptation, FastDraw eliminates the need for hand-tuned heuristics and expensive manual labeling for new environments. While the current model assumes lanes are functions of the vertical axis—which may struggle with extreme U-turns—it represents a significant step toward faster, more robust lane detection for real-world autonomous driving.

Summary updated

Abstract

The search for predictive models that generalize to the long tail of sensor inputs is the central difficulty when developing data-driven models for autonomous vehicles. In this paper, we use lane detection to study modeling and training techniques that yield better performance on real world test drives. On the modeling side, we introduce a novel fully convolutional model of lane detection that learns to decode lane structures instead of delegating structure inference to post-processing. In contrast to previous works, our convolutional decoder is able to represent an arbitrary number of lanes per image, preserves the polyline representation of lanes without reducing lanes to polynomials, and draws lanes iteratively without requiring the computational and temporal complexity of recurrent neural networks. Because our model includes an estimate of the joint distribution of neighboring pixels belonging to the same lane, our formulation includes a natural and computationally cheap definition of uncertainty. On the training side, we demonstrate a simple yet effective approach to adapt the model to new environments using unsupervised style transfer. By training FastDraw to make predictions of lane structure that are invariant to low-level stylistic differences between images, we achieve strong performance at test time in weather and lighting conditions that deviate substantially from those of the annotated datasets that are publicly available. We quantitatively evaluate our approach on the CVPR 2017 Tusimple lane marking challenge, difficult CULane datasets [8], and a small labeled dataset of our own and achieve competitive accuracy while running at 90 FPS.
BibTeX
@inproceedings{philion2019fastdraw,
  author = {Jonah Philion},
  title = {{FastDraw: Addressing the Long Tail of Lane Detection by Adapting a Sequential Prediction Network.}},
  booktitle = {Workshops @ CVPR},
  year = {2019},
}
Figure