Computational modeling of depth ordering in occlusion through accretion or deletion of texture

Harald Ruda, Gennady Livitz, Guillaume Riesen, Ennio MingollaJournal of Vision 2015 · DOI

TL;DR

This paper presents an updated computational model, the FormMotionOcclusion (FMO) model, that successfully explains how humans perceive depth order from motion cues like accretion and deletion. It addresses counterintuitive phenomena like the Moonwalk illusion and dynamic depth changes in textured displays, demonstrating the crucial interplay between motion and boundary processing.

What Problem Does This Paper Address?

Our visual system is remarkably adept at determining which objects are in front of others, a fundamental aspect of scene understanding. A key cue for this is "accretion and deletion" (AD) of texture, where new texture appearing (accretion) or disappearing (deletion) at an edge signals that one surface is moving in front of another. Classically, AD implies the occluded surface is farther away. However, recent illusions like the "Moonwalk illusion" challenge this, showing AD can sometimes lead to the occluding surface appearing closer. Furthermore, dynamic depth changes, as seen in textured versions of the Michotte rabbit hole phenomenon, require models to handle complex, time-varying depth ordering. This paper aims to reconcile these observations by proposing a computational model that can explain both classical AD phenomena and these more complex, counterintuitive scenarios.

Methods

The authors present an updated version of their previous ForMotionOcclusion (ForMOcclusion) model, now called the FormMotionOcclusion (FMO) model. This model is built upon the Barlow-Levick principle of motion detection and features two interacting streams:

  • MOTION stream: Detects elementary motion signals and, crucially, accretion and deletion events. It's designed to be more robust to grayscale textures than previous versions.
  • FORM stream: Detects and completes boundaries. It receives input from the MOTION stream to help define these boundaries.

A key innovation is the introduction of push-pull signals. These signals are generated by AD events and, when combined with boundary information from the FORM stream, create a clear depth differential. If no boundary is present (as in some ambiguous cases), these signals tend to cancel out, leading to weaker depth perception. The model also incorporates faster diffusion processes for filling in depth information and a more biologically plausible competition between depth pools.

Datasets

The paper does not explicitly mention training or evaluation on large, pre-existing datasets. Instead, it relies on:

  • Simulated stimuli: The authors generated specific visual stimuli to test their model, including:
  • Classical accretion and deletion scenarios (static and moving boundaries).
  • The Moonwalk illusion stimulus.
  • A textured version of the Michotte rabbit hole phenomenon.
  • A "shear" stimulus (motion parallel to the boundary).
  • A "Great Roe stimulus" with identical motion statistics but different textures.
  • Reference to existing experimental data: The model's performance is compared against human perception data from previous studies (e.g., Kaplan, 1969; Royden et al., 1988; Kromrey et al., 2011).

Key Insights and Ideas

  • Interplay of Motion and Boundaries is Crucial: The FMO model highlights that AD cues alone are not always sufficient for depth ordering. The interaction with boundary formation (from the FORM stream) is essential for generating strong depth percepts.
  • Push-Pull Signals Explain Ambiguity: The introduction of push-pull signals provides a mechanism to explain how AD can lead to both near and far depth percepts, and how these signals can cancel out in ambiguous situations (like the Moonwalk illusion's surround).
  • Grayscale Textures are Handled: Modifications to the MOTION stream allow the model to process grayscale textures, making it more applicable to realistic imagery.
  • Dynamic Depth is Explained: The model can account for depth that changes over time, as demonstrated by the textured Michotte rabbit hole effect, where surfaces appear to bend in depth.
  • Moonwalk Illusion Explained by Lack of Clear Boundary: The model explains the Moonwalk illusion by showing that the flickering surround generates spurious AD signals, preventing the FORM stream from creating a clear boundary. This ambiguity leads to the central moving area being perceived as closer.

Key Results

The paper demonstrates the FMO model's ability to:

  • Reproduce classical AD phenomena: Simulations show correct depth ordering for static and moving boundary accretion/deletion stimuli (Figures 10 & 11).
  • Explain the Moonwalk illusion: The model successfully predicts the counterintuitive percept where the central moving area appears in front of the flickering surround (Figures 14 & 15).
  • Account for textured Michotte rabbit hole effect: The model generates dynamic depth ordering consistent with the percept of a circle sliding into a slit (Figures 16 & 17).
  • Handle shear stimuli: The model correctly predicts that faster moving areas appear nearer, even without AD cues (Figure 13).

While quantitative metrics like accuracy are not the focus, the qualitative results show a strong correspondence between the model's output and human perceptual judgments for these complex stimuli.

Limitations and Future Work

  • Boundary Shape Ignored: The current FMO model lacks a mechanism to incorporate boundary shape (concavity/convexity) into depth ordering, which the authors suggest is an important factor in human perception.
  • Boundary Flow (BF) Cues Not Used: The model primarily relies on Accretion/Deletion (AD) cues and ignores Boundary Flow (BF) cues, which are also important for depth perception. The authors suggest combining their model with BF-based models for a more comprehensive understanding.
  • Stereopsis Not Included: The model does not incorporate stereoscopic depth cues, which are crucial in natural vision.
  • Texture Density: The simulations use dense random dot textures, which may not fully represent natural scenarios where textures are sparser and other cues (contrast, color) play a larger role.
  • "Great Roe Stimulus" Investigation Ongoing: The authors are continuing to investigate the "Great Roe stimulus" to understand the role of motion signals versus other cues in boundary perception when textures are statistically identical but not the same.

Future work could involve integrating BF cues, stereopsis, and exploring how sparser textures and other visual cues interact with AD signals.

Summary updated

Abstract

Understanding the depth ordering of surfaces in the natural world is one of the most fundamental operations of the primate visual system. Surfaces that undergo accretion or deletion (AD) of texture are always perceived to behind an adjacent surface. An updated ForMotionOcclusion model (Barnes & Mingolla, 2013) includes two streams for computing motion signals and boundary signals. The two streams generate depth percepts such that AD signals together with boundary signals generate a farther depth on the occluded side of the boundary. The model fits the classical data (Kaplan, 1969) as well as the observation that moving surfaces tend to appear closer in depth (Royden, Baker, & Allman, 1988), for both binary and grayscale stimuli. The recent "Moonwalk illusion" described by Kromrey, Bart, and Hegdé (2011) upends the classical view that the surface undergoing AD always becomes the background. Here the surface that undergoes AD appears to be in front of the surrounding surface-a result of the random flickering noise in the surround. As an additional challenge, we developed an AD display with dynamic depth ordering. A new texture version of the Michotte rabbit hole phenomenon (Michotte, Thinès, & Crabbé, 1964/1991) generates depth that changes in part of the display area. Because the ForMotionOcclusion model separates the computation of boundaries from the computation of AD signals, it is able to explain the counterintuitive Moonwalk stimulus. We show simulations that explain the workings of the model and how the model explains the Moonwalk and textured Michotte phenomena.
BibTeX
@article{ruda2015computational,
  author = {Harald Ruda and Gennady Livitz and Guillaume Riesen and Ennio Mingolla},
  title = {{Computational modeling of depth ordering in occlusion through accretion or deletion of texture}},
  journal = {Journal of Vision},
  year = {2015},
  doi = {10.1167/15.9.20},
}
Figure