Fast Road Segmentation via Uncertainty-aware Symmetric Network

Yicong Chang, Feng Xue, Fei Sheng, Wenteng Liang, Anlong Ming

Introduction

Road segmentation—the precise pixel-level identification of drivable surfaces—is a fundamental requirement for autonomous vehicle navigation. While modern deep learning models have made significant strides in this area, achieving a balance between high-speed performance and robust accuracy remains a challenge. Traditional methods often struggle with environmental variability, such as dramatic lighting changes or sensor-specific noise.

To address these issues, researchers have introduced USNet (Uncertainty-aware Symmetric Network). This novel architecture leverages evidence theory to intelligently fuse RGB and depth (D) data, achieving state-of-the-art accuracy while maintaining real-time processing speeds of over 43 frames per second (FPS).

The Challenge of Multi-Modal Fusion

Integrating color (RGB) and depth information is a common strategy to improve segmentation reliability. However, existing RGB-D methods typically face two significant hurdles:

  1. Computational Latency: High-accuracy fusion models are often computationally intensive, making them difficult to deploy in self-driving systems that require millisecond-level response times.
  2. Feature Conflict: RGB and depth sensors possess distinct vulnerabilities. RGB sensors are susceptible to overexposure and deep shadows, while depth sensors (often derived from LiDAR or stereo cameras) can be noisy or inaccurate at long distances. Simple fusion techniques, such as concatenating or adding features early in the network, can allow misleading information from one sensor to "pollute" the other, leading to segmentation errors.

USNet: Architecture and Methodology

USNet departs from standard fusion paradigms by adopting a symmetric, uncertainty-driven approach designed to preserve the integrity of each sensor modality until the final decision stage.

Symmetric Unimodal Subnetworks

Instead of merging RGB and depth data in the early layers, USNet utilizes two independent, lightweight branches based on the ResNet-18 architecture. This separation ensures that the network extracts distinct features from each modality without the risk of early-stage feature conflict.

Multi-scale Evidence Collection (MEC)

At the end of each unimodal branch, the model employs a Multi-scale Evidence Collection (MEC) module. Using atrous (dilated) convolutions, the MEC gathers "evidence" for road versus non-road classification across various spatial scales, ensuring the model captures both fine-grained details and broader contextual information.

Uncertainty-aware Fusion (UAF)

The core innovation of USNet is the Uncertainty-aware Fusion (UAF) module, which is grounded in Subjective Logic—a formalization of Dempster-Shafer evidence theory. Unlike standard neural networks that output simple probabilities (e.g., a 70% chance a pixel is a road), USNet calculates a "belief mass" and an "uncertainty mass" for every pixel from both sensors.

This allows for Smart Arbitration: if the RGB branch is highly uncertain about a specific pixel due to a dark shadow, the UAF module automatically assigns more weight to the depth branch's prediction. Conversely, if the depth data is unreliable due to distance or sensor noise, the model relies more heavily on the RGB evidence.

Efficiency via Lightweight Design

To maintain high throughput, the researchers implemented a "zero-parameter decoder." By utilizing simple upsampling and summation rather than heavy convolutional layers for the final reconstruction, the model significantly reduces floating-point operations (FLOPs) without sacrificing segmentation quality.

Experimental Results

USNet was rigorously evaluated on the KITTI Road and Cityscapes datasets, two industry-standard benchmarks for urban and highway scene understanding.

  • Real-Time Speed: USNet achieves a processing speed of 43.6 FPS on an NVIDIA GTX 1080Ti. This is approximately ten times faster than previous state-of-the-art RGB-D models like PLARD or SNE-RoadSeg, making it viable for actual vehicle deployment.
  • High Accuracy: On the KITTI benchmark, the model reached a MaxF score of 96.89%. It outperformed all other RGB-only methods and the majority of existing RGB-D and LiDAR-based approaches.
  • Robustness: Qualitative analysis demonstrates that USNet effectively ignores tree shadows that typically confuse RGB-only models and filters out distant artifacts that often mislead depth-only systems.

Conclusion and Future Work

USNet represents a significant step forward in real-time road segmentation by treating sensor fusion as an evidence-based decision process rather than a simple feature merge. By explicitly modeling uncertainty, the network achieves a level of robustness essential for safe autonomous driving.

While highly effective, the current implementation relies on a Surface Normal Estimator (SNE) to preprocess raw depth data, which adds a step to the pipeline. Future research may focus on integrating this preprocessing directly into the end-to-end learning framework. Additionally, expanding the model from binary road segmentation to multi-class semantic segmentation (detecting pedestrians, vehicles, and signage) remains a promising avenue for further development.

Summary updated

Abstract

The high performance of RGB-D based road segmentation methods contrasts with their rare application in commercial autonomous driving, which is owing to two reasons: 1) the prior methods cannot achieve high inference speed and high accuracy in both ways; 2) the different properties of RGB and depth data are not well-exploited, limiting the reliability of predicted road. In this paper, based on the evidence theory, an uncertainty-aware symmetric network (USNet) is proposed to achieve a trade-off between speed and accuracy by fully fusing RGB and depth data. Firstly, cross-modal feature fusion operations, which are indispensable in the prior RGB-D based methods, are abandoned. We instead separately adopt two light-weight subnetworks to learn road representations from RGB and depth inputs. The light-weight structure guarantees the real-time inference of our method. Moreover, a multiscale evidence collection (MEC) module is designed to collect evidence in multiple scales for each modality, which provides sufficient evidence for pixel class determination. Finally, in uncertainty-aware fusion (UAF) module, the uncertainty of each modality is perceived to guide the fusion of the two subnetworks. Experimental results demonstrate that our method achieves a state-of-the-art accuracy with real-time inference speed of 43+ FPS. The source code is available at https://github.com/morancyc/USNet.
BibTeX
@inproceedings{chang2022fast,
  author = {Yicong Chang and Feng Xue and Fei Sheng and Wenteng Liang and Anlong Ming},
  title = {{Fast Road Segmentation via Uncertainty-aware Symmetric Network}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2022},
  url = {https://openreview.net/forum?id=ktXtHYSblHg},
}
Figure