A Continuous Occlusion Model for Road Scene Understanding

Vikas Dhiman, Quoc-Huy Tran, Jason J. Corso, Manmohan ChandrakerCVPR 2016 · DOI

Abstract

Accurate scene understanding in autonomous driving is often hindered by occlusions, where objects block one another from the camera's view. Traditional methods relying on 2D masks or motion segmentation frequently fail when objects are static or move at similar velocities. This paper introduces a physically-inspired 3D model that treats objects as continuous, translucent ellipsoids. By calculating reflection and transmission probabilities for camera rays based on the Beer-Lambert law, the model provides a unified framework for associating point tracks with objects and improving 3D localization. Experimental results on the KITTI dataset show that this approach significantly outperforms existing baselines, particularly in scenarios involving static parked cars and complex occlusions.

Introduction

In the context of computer vision for road scenes, occlusions represent a major hurdle. When one object partially or fully blocks another, the resulting loss of information makes it difficult to track points or localize objects accurately in 3D space. Existing solutions typically fall into two categories: 2D-based methods using discrete masks, which lack physical depth, and motion segmentation, which relies on velocity differences. However, in urban environments, many objects like parked cars are static, causing motion-based methods to fail.

To address these limitations, we propose a continuous, fully 3D model that is physically interpretable and robust to varying object dynamics.

The Continuous Occlusion Model

The core innovation of this work is representing objects as translucent 3D ellipsoids rather than rigid boxes. Instead of a "hard" binary boundary, an object's occupancy is defined by a continuous logistic function. This ensures that the model remains differentiable, facilitating its integration into standard optimization frameworks used in robotics and mapping.

Overview of the Continuous Occlusion Model pipeline, from SFM inputs to probabilistic object association.

Physics-Inspired Probabilities

The model determines the likelihood of observing a specific point in space by analyzing two primary factors:

  1. Reflection Probability ($P_{reflection}$): The likelihood that a camera ray hits the "surface" of an object, calculated using the gradient of the occupancy function.
  2. Transmission Probability ($P_{transmission}$): Inspired by the Beer-Lambert law from optics, this calculates the likelihood that a ray passes through free space or the translucent outer layers of other objects before reaching the target.
Probability of reflection and transmission as a function of depth from the camera.

By combining these probabilities, the system can perform "soft assignments," providing a probability that a point track belongs to a specific car rather than a binary yes/no. This is particularly effective for handling fuzzy boundaries where objects overlap in the camera's view.

Experimental Evaluation

The model was evaluated using the KITTI raw dataset, encompassing 35 sequences with over 10,000 frames and 634 individual object tracks. The proposed model was compared against state-of-the-art motion segmentation methods (RAS and BM) and a heuristic bounding-box (BBox) baseline.

Key Results

  • Point Track Association: The proposed model achieved a mean association error of 8.6%, a marked improvement over the bounding-box baseline (12.6%) and motion segmentation methods like BM (21.9%).
  • Occlusion Handling: In scenarios involving occluded dynamic objects, the model's error rate was 13.2%, nearly half that of the best motion segmentation baseline (30.1%).
  • 3D Localization: By incorporating occlusion reasoning into energy functions, the model reduced translation error to 3.24 meters, compared to 5.61 meters for initial estimates.

Conclusion and Future Work

The continuous occlusion model provides a robust mathematical foundation for 3D scene understanding by treating objects as probabilistic volumes. While the current implementation faces computational bottlenecks—taking several minutes per frame window—future work will focus on speed optimizations using simpler graphical model structures. Additionally, moving beyond simple ellipsoids to multi-sphere representations will allow for more precise modeling of complex vehicle geometries while maintaining the benefits of a continuous framework.

Summary updated

BibTeX
@inproceedings{dhiman2016continuous,
  author = {Vikas Dhiman and Quoc-Huy Tran and Jason J. Corso and Manmohan Chandraker},
  title = {{A Continuous Occlusion Model for Road Scene Understanding}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2016},
  doi = {10.1109/cvpr.2016.469},
}
Figure