Local Detection of Stereo Occlusion Boundaries.

Jialiang Wang, Todd E. ZicklerCVPR 2019 · OpenReview

Abstract

Stereo occlusion boundaries are one-dimensional structures in the visual field that separate foreground regions of a scene that are visible to both eyes (binocular regions) from background regions of a scene that are visible to only one eye (monocular regions). Stereo occlusion boundaries often coincide with object boundaries, and localizing them is useful for tasks like grasping, manipulation, and navigation. This paper describes the local signatures for stereo occlusion boundaries that exist in a stereo cost volume, and it introduces a local detector for them based on a simple feedforward network with relatively small receptive fields. The local detector produces better boundaries than many other stereo methods, even without incorporating explicit stereo matching, top-down contextual cues, or single-image boundary cues based on texture and intensity.

Introduction

In stereo vision, "occlusion boundaries" are the 1D curves where the view of the background is cut off for one of the two cameras. These areas are notoriously difficult for computer vision algorithms. While modern deep learning has slashed error rates in "binocular regions" (visible to both eyes) to around 7%, error rates in "monocular regions" (visible to only one eye) remain stubbornly high, often exceeding 50%.

Comparison of error rates in binocular versus monocular regions.

Most existing methods try to find these boundaries using global consistency checks or by assuming that depth edges must align with high-contrast texture edges in the image. However, these approaches fail when textures are absent or when the scene geometry is complex. This paper investigates whether these boundaries can be detected using only local information from the stereo matching process itself.

Methodology: Local Signatures in Cost Volumes

The authors propose a local detector based on a 3D variant of the Holistically-Nested Edge Detection (HED) architecture. The core insight is that stereo occlusion boundaries leave "fingerprints" in the cost volume that are locally detectable.

Geometry of stereo occlusion and its manifestation in the cost volume.

The system processes a multiscale stereo cost volume $C(x, y, d, s)$, which stores matching costs for every pixel at various disparities and window sizes. To enhance detection, geometric "shearing" is applied to the cost volume to align potential occlusion rays. A cumulative minimum operation along the disparity dimension helps pool evidence from background points. The resulting volume is processed by a feedforward CNN. Notably, the model is trained entirely on synthetic, abstract data—simple scenes with two planes at different depths—forcing it to learn fundamental geometry rather than specific image features.

Key Results

The proposed method achieved an ODS F-score of 0.61 on the Middlebury benchmark, significantly outperforming established methods like SGM-LR (0.32) and PSM-Net (0.37).

Key findings include:

  • Robustness to Texture: The detector succeeded on perceptual stimuli (like random-dot stereograms) where traditional algorithms failed completely.
  • Precision: Qualitative results show that the boundaries produced are much "sharper" and more accurately placed than those extracted from dense disparity maps generated by other state-of-the-art stereo networks.
  • Ignoring the Ordering Constraint: By ignoring the assumption that the relative order of points doesn't change between views, the model correctly detects boundaries even for very thin objects.

Discussion and Future Work

The authors acknowledge that some occlusion boundaries are locally indistinguishable from simple texture edges. While this can lead to false positives (such as E and F-type points in their taxonomy), it also reflects human perception, where "depth-events" are identified regardless of the underlying cause.

Future work includes making the filters orientation-aware to improve performance and integrating this standalone local detector into larger, global stereo matching systems to enhance overall depth estimation in monocular regions.

Summary updated

Abstract

Stereo occlusion boundaries are one-dimensional structures in the visual field that separate foreground regions of a scene that are visible to both eyes (binocular regions) from background regions of a scene that are visible to only one eye (monocular regions). Stereo occlusion boundaries often coincide with object boundaries, and localizing them is useful for tasks like grasping, manipulation, and navigation. This paper describes the local signatures for stereo occlusion boundaries that exist in a stereo cost volume, and it introduces a local detector for them based on a simple feedforward network with relatively small receptive fields. The local detector produces better boundaries than many other stereo methods, even without incorporating explicit stereo matching, top-down contextual cues, or single-image boundary cues based on texture and intensity.
BibTeX
@inproceedings{wang2019local,
  author = {Jialiang Wang and Todd E. Zickler},
  title = {{Local Detection of Stereo Occlusion Boundaries.}},
  booktitle = {CVPR},
  year = {2019},
  url = {https://openreview.net/forum?id=r7AVWkmlu6S},
}
Figure