Abstract
Lane detection is extremely important for autonomous vehicles. For this reason, many approaches use lane boundary information to locate the vehicle inside the street, or to integrate GPS-based localization. As many other computer vision based tasks, convolutional neural networks (CNNs) represent the state-of-the-art technology to identify lane boundaries. However, the position of the lane boundaries with respect to the vehicle may not suffice for reliable positioning; for path planning or localization, information regarding lane types may also be needed. In this work, we present an end-to-end system for lane boundary identification, clustering, and classification, based on two cascaded neural networks, that runs in real-time. To build the system, 14,336 lane boundaries instances of the TuSimple dataset for lane detection have been labeled using 8 different classes. Our dataset and the code for inference are available online.
Introduction
Most existing lane detection systems focus solely on localization—identifying the pixels or coordinates where a lane exists. However, for a self-driving car to make a decision (like "Can I legally overtake this vehicle?"), it needs to know the lane type. A dashed line allows a lane change, while a solid double yellow line does not. The challenge lies in doing this efficiently. Many "instance segmentation" methods require heavy post-processing or clustering algorithms that are too slow for high-speed driving. Furthermore, there is a lack of large-scale datasets that include both the position and the specific classification of lane boundaries.
Methodology
The authors propose a two-stage "cascaded" architecture to achieve real-time performance while providing semantic classification of lane types.
Instance Segmentation
The first stage uses ERFNet to identify up to four lane boundaries. Instead of simple binary masking, it employs a loss function based on Kullback-Leibler (KL) divergence. This enables the network to group pixels into distinct lane instances directly, bypassing slow post-processing clustering. To ensure training stability, the authors used curriculum learning, transitioning from simple binary classification to the more complex KL loss.
Descriptor Extraction and Classification
For every detected lane, the system samples a fixed number of points and rearranges them into a small, square "descriptor" image. This process effectively "straightens" the lane, making the subsequent classification independent of the vehicle's relative position or the lane's curvature.
These descriptors are then fed into a lightweight CNN (a derivative of H-Net) that classifies the lane into one of eight categories, such as single white continuous, dashed, or Botts' dots.
Datasets
The study utilizes the TuSimple dataset, a popular benchmark for lane detection. Since the original dataset only provided lane positions, the authors manually labeled 14,336 lane instances with 8 different classes. These annotations and the inference code have been made publicly available.
Key Results
The system demonstrates impressive efficiency and accuracy:
- Speed: It runs at 58.93 FPS on an NVIDIA Titan Xp, suitable for real-time use.
- Segmentation Accuracy: On the TuSimple benchmark, the model achieved 95.24% accuracy.
- Classification Accuracy: The system achieved 96-97% accuracy in distinguishing between dashed and continuous lines. Even when adding a third class (double-dashed), accuracy remained high at 96%.
Conclusion
By converting curving lanes into standardized square descriptors, the network avoids the need to learn patterns at every possible angle. While the current architecture is limited to four lane boundaries and was tested on high-end hardware, it represents a significant step toward real-time semantic lane understanding. Future work may involve increasing dataset diversity for rare lane types and optimizing for embedded automotive hardware.
Summary updated