Robust Drivable Road Region Detection for Fixed-Route Autonomous Vehicles Using Map-Fusion Images

Yichao Cai, Dachuan Li, Xiao Zhou, Xingang Mou

TL;DR

This paper introduces a novel approach for detecting drivable road regions for autonomous vehicles by fusing camera images with local route map information. This "map-fusion" technique significantly improves detection accuracy and robustness, especially in challenging scenarios like intersections and areas with poor lane markings, by providing crucial contextual guidance.

What Problem Does This Paper Address?

Autonomous vehicles rely heavily on accurate environment perception to navigate safely. A critical component of this is "drivable road region detection" – identifying the safe, traversable areas of the road. However, existing methods struggle with complex real-world conditions such as:

  • Unstructured road environments: Missing or vague lane markings and road borders.
  • Variable illumination: Shadows, glare, and changing light conditions.
  • Ununiform pavement: Different textures and colors on the road surface.
  • Occlusions: Objects blocking the view of the road.
  • Intersections and multi-lane roads: Complex layouts with multiple potential paths.

These challenges make it difficult for traditional computer vision techniques, which often rely solely on visual features, to reliably determine where a vehicle can safely drive.

Methods

The proposed approach combines computer vision with route map data and a deep neural network. Here's a breakdown:

  1. Inverse Perspective Mapping (IPM): The raw front-view images from the vehicle's camera are transformed into a bird's-eye view. This is done using a pre-calibrated projection matrix to reduce redundant information and make road features more prominent.
  2. Map-Fusion Image (MFI) Generation:
    • Local Route Map Extraction: Using the vehicle's GPS location and a global route map, a local route map segment corresponding to the current camera view is extracted. This map provides information about the intended path, including lane guidance and driving direction.
  3. Fusion: The extracted local route map is overlaid onto the bird's-eye view image. This creates the Map-Fusion Image (MFI), where the visual information from the camera is augmented with explicit navigational cues from the route map.
  4. Drivable Road Region Detection: A Fully Convolutional Network (FCN) based on the VGG16 architecture (FCN-VGG16) is employed. This network is trained on the generated MFIs to perform semantic segmentation, classifying each pixel as either drivable or non-drivable. The FCN-VGG16 architecture uses a CNN encoder to extract features and a decoder with up-sampling layers to generate a pixel-wise segmentation map.

The key novelty lies in the map-fusion step, which provides the FCN with contextual information that visual features alone lack, especially in ambiguous situations.

Datasets

The paper utilizes real-world driving scenario videos captured by a camera mounted on a testing bus. This dataset includes:

  • A 4.3 km testing track.
  • Sections with straight roads, curved roads, and intersections.
  • Scenarios with varying illumination, pavement appearance, and the presence of other road users (pedestrians, vehicles, motorcycles, bicycles).
  • Corresponding GPS data for localization.

The authors do not explicitly mention using publicly available datasets for training or evaluation, suggesting their experiments were conducted on their custom-collected data.

Key Insights and Ideas

  • Route Map as Contextual Heuristic: The most significant insight is that route map information acts as a powerful "heuristic" for drivable road detection. It guides the perception system, especially when visual cues are unreliable.
  • Complementary Information: Visual data is rich in texture and detail, while route maps provide semantic context (where to go). Fusing them creates a more comprehensive understanding of the driving environment.
  • Robustness through Fusion: The map-fusion approach demonstrably improves robustness against common perception challenges like poor lighting, absent lane markings, and varied road surfaces.
  • Fixed-Route Advantage: The approach is specifically tailored for fixed-route vehicles (shuttles, buses), where route map availability is a given and simplifies the problem compared to fully autonomous navigation in any environment.

Key Results

  • Improved Accuracy: The proposed map-fusion approach achieved an average detection accuracy (Pixel Accuracy - PA) of 0.917, significantly outperforming the conventional FCN-based approach using raw images without map-fusion, which achieved 0.811.
  • Robustness Demonstrated: Experiments showed that the approach maintains high accuracy even in complex scenarios:
  • Intersections and turning segments with absent lane markings.
  • Variable illumination conditions (shadows).
  • Undesirable pavement appearance (cracks, inconsistent colors).
  • Presence of other road users (pedestrians, vehicles, motorcycles, bicycles).
  • Errors in local map extraction or IPM conversions.
  • Effective Feature Extraction: The FCN-VGG16 model, trained on MFIs, effectively extracts drivable road regions.

Limitations and Future Work

  • Limitations:
    • The approach relies on the availability and accuracy of route maps and GPS data.
  • The IPM projection matrix is fixed and pre-calibrated, which might be less robust to significant vehicle roll or pitch movements (though the authors argue it's acceptable for low-speed driving).
  • The current implementation is focused on fixed-route vehicles.
  • Future Work:
    • Incorporating other sensor data: Integrating additional vehicle location data, such as LiDAR point-cloud matching, to further enhance robustness.
  • More comprehensive investigation: Expanding testing to a wider range of real-world driving scenarios.
  • Implementation and integration: Developing the system for integration into an actual autonomous driving system.
  • Generalization: While tested on fixed-route vehicles, exploring how to adapt this approach for more general autonomous driving scenarios could be a future direction.

Summary updated

Abstract

Environment perception is one of the major issues in autonomous driving systems. In particular, effective and robust drivable road region detection still remains a challenge to be addressed for autonomous vehicles in multi-lane roads, intersections and unstructured road environments. In this paper, a computer vision and neural networks-based drivable road region detection approach is proposed for fixed-route autonomous vehicles (e.g., shuttles, buses and other vehicles operating on fixed routes), using a vehicle-mounted camera, route map and real-time vehicle location. The key idea of the proposed approach is to fuse an image with its corresponding local route map to obtain the map-fusion image (MFI) where the information of the image and route map act as complementary to each other. The information of the image can be utilized in road regions with rich features, while local route map acts as critical heuristics that enable robust drivable road region detection in areas without clear lane marking or borders. A neural network model constructed upon the Convolutional Neural Networks (CNNs), namely FCN-VGG16, is utilized to extract the drivable road region from the fused MFI. The proposed approach is validated using real-world driving scenario videos captured by an industrial camera mounted on a testing vehicle. Experiments demonstrate that the proposed approach outperforms the conventional approach which uses non-fused images in terms of detection accuracy and robustness, and it achieves desirable robustness against undesirable illumination conditions and pavement appearance, as well as projection and map-fusion errors.
BibTeX
@inproceedings{cai2018robust,
  author = {Yichao Cai and Dachuan Li and Xiao Zhou and Xingang Mou},
  title = {{Robust Drivable Road Region Detection for Fixed-Route Autonomous Vehicles Using Map-Fusion Images}},
  booktitle = {Sensors},
  year = {2018},
  url = {https://openreview.net/forum?id=m3FpethaDa},
}
Figure