Weakly But Deeply Supervised Occlusion-Reasoned Parametric Road Layouts

Buyu Liu, Bingbing Zhuang, Manmohan Chandraker

Abstract

We propose an end-to-end network that takes a single perspective RGB image of a complex road scene as input, to produce occlusion-reasoned layouts in perspective space as well as a parametric bird's-eye-view (BEV) space. In contrast to prior works that require dense supervision such as semantic labels in perspective view, our method only requires human annotations for parametric attributes that are cheaper and less ambiguous to obtain. To solve this challenging task, our design is comprised of modules that incorporate inductive biases to learn occlusion-reasoning, geometric transformation and semantic abstraction, where each module may be supervised by appropriately transforming the parametric annotations. We demonstrate how our design choices and proposed deep supervision help achieve meaningful representations and accurate predictions. We validate our approach on two public datasets, KITTI and NuScenes, to achieve state-of-the-art results with considerably less human supervision.

Introduction

For autonomous vehicles to navigate safely, understanding the road layout—including areas occluded by vehicles or buildings—is critical. Most contemporary models generate bird’s-eye-view (BEV) maps but face two significant hurdles: high annotation costs and occlusion handling. Training typically requires "dense" supervision, where humans label every pixel, a process taking roughly 20 minutes per image. Furthermore, standard models struggle to "hallucinate" road features hidden behind foreground objects in the perspective view.

Comparison of the proposed method using cheap parametric supervision versus conventional methods requiring expensive dense labels.

This paper introduces an end-to-end deep learning framework that predicts parametric road layouts (e.g., lane counts, sidewalk presence) from a single RGB image. By utilizing a modular design that reasons about geometry and occlusions, the authors achieve state-of-the-art results using only "cheap" attribute-level annotations (~20 seconds per image), bypassing the need for expensive pixel-level labeling.

Methodology

The proposed architecture consists of a three-stage pipeline incorporating "inductive biases"—architectural shortcuts that guide the model toward learning geometric logic rather than simple pattern memorization.

  1. Perspective Semantics (PS) Module: This module processes the raw RGB image to predict a semantic map. It is specifically trained to be "occlusion-reasoned," attempting to predict road and sidewalk features even when obscured by vehicles.
  2. Top-view Semantics (TS) Module: This module transforms the perspective map into a bird's-eye view. It employs a geometric transformation (assuming a flat road plane) and a Hallucination Module to fill gaps caused by the camera's limited field of view or distant data.
  3. Top-view Parametric Prediction (TPP) Module: The final stage regresses attributes from the hallucinated BEV map, including binary (one-way status), multi-class (lane count), and continuous (distance to side roads) values.
The modular architecture of the proposed model, featuring Perspective Semantics (PS), Top-view Semantics (TS), and Top-view Parametric Prediction (TPP) modules.

Deep Supervision via Rendering: To train intermediate modules without pixel-level labels, the authors utilize a renderer that converts parametric attributes into "pseudo-ground-truth" pixel masks. This allows the PS and TS modules to receive supervision derived directly from the cheap human annotations.

Experimental Results

The model was evaluated on the KITTI and NuScenes benchmarks. Key findings include:

  • Efficiency and Accuracy: Despite using significantly simpler training data, the method achieves results comparable to "Strongly Supervised" models. On KITTI, the model reached an F1 score of 0.473, a massive improvement over the standard RGB-to-parametric baseline (0.176).
  • Complex Scene Handling: On the challenging NuScenes dataset, the model maintained strong performance (0.130 F1) amidst complex intersections and heavy traffic.
  • Reasoning and Hallucination: Qualitative analysis confirms the model's ability to infer road layouts behind parked cars and around sharp curves, demonstrating effective occlusion reasoning.

Discussion and Future Work

The research demonstrates that weak supervision is sufficient for learning complex road geometries if the model is provided with a proper geometric framework. By forcing the model to predict intermediate perspective and BEV maps, the learning process becomes more stable.

However, the current "flat world assumption" for geometric transformation may limit performance on steep terrain. Future work could extend this framework to video sequences for improved spatio-temporal consistency or explore training on synthetic data, given the parametric nature of the output.

Summary updated

Abstract

We propose an end-to-end network that takes a single perspective RGB image of a complex road scene as input, to produce occlusion-reasoned layouts in perspective space as well as a parametric bird's-eye-view (BEV) space. In contrast to prior works that require dense supervision such as semantic labels in perspective view, our method only requires human annotations for parametric attributes that are cheaper and less ambiguous to obtain. To solve this challenging task, our design is comprised of modules that incorporate inductive biases to learn occlusion-reasoning, geometric transformation and semantic abstraction, where each module may be supervised by appropriately transforming the parametric annotations. We demonstrate how our design choices and proposed deep supervision help achieve meaningful representations and accurate predictions. We validate our approach on two public datasets, KITTI and NuScenes, to achieve state-of-the-art results with considerably less human supervision.
BibTeX
@inproceedings{liu2022weakly,
  author = {Buyu Liu and Bingbing Zhuang and Manmohan Chandraker},
  title = {{Weakly But Deeply Supervised Occlusion-Reasoned Parametric Road Layouts}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2022},
  url = {https://openreview.net/forum?id=w-V4Tp55YDS},
}
Figure