Disparity statistics in natural scenes

Yi Liu, Alan C. Bovik, Lawrence K. CormackJournal of Vision 2008 · DOI

TL;DR

This paper quantifies the distribution of binocular disparities in natural outdoor and indoor scenes, revealing that suprathreshold disparities are common even at far distances. This challenges the notion that stereopsis is limited to near-field vision and suggests that our visual system is well-equipped to handle depth perception in a much wider range of environments than previously assumed.

What Problem Does This Paper Address?

Stereopsis, our ability to perceive depth using the slight differences in the images seen by our two eyes (binocular disparity), is a fundamental aspect of human vision. However, a key question has been: what are the actual disparities encountered in natural environments, and how do they relate to the capabilities of our visual system? It's widely believed that stereopsis is primarily effective only within a few meters because retinal disparity decreases significantly with distance, and cues like convergence and accommodation become less reliable. This paper aims to address this by quantitatively measuring binocular disparities in natural scenes and comparing them to psychophysical and neurophysiological data.

Methods

The researchers employed a multi-step approach:

  1. Range Data Acquisition: They utilized existing high-resolution 3D range maps of forest and indoor scenes. These maps provide precise distance information for various points in the scene.
  2. Eye Model and Fixation Simulation: To convert scene distances into retinal disparities, they used a simplified model of the human eye (a perfect sphere with a given interpupillary distance). Crucially, they needed to determine where a human observer would be looking (fixation point). For forest scenes, they conducted an experiment where participants walked in a park and reported their fixation points, which were then measured. For indoor scenes, they simulated fixation points based on the assumption that observers would fixate on the nearest object in a random direction.
  3. Disparity Computation: Using the range data, eye model, and fixation points, they calculated the horizontal binocular disparity for each point in the scene relative to the fixation point. This involved projecting scene points onto a simplified retinal plane and calculating the angular difference.
  4. Statistical Analysis: They then analyzed the distributions of these computed disparities, looking at their shape, mean, standard deviation, and the prevalence of suprathreshold disparities (those greater than 10 arcsec). They also examined how these distributions varied with eccentricity (the angle from the center of gaze).

Datasets

  • Forest Scene Range Maps: Data from 23 forest scenes collected by Yang and Purves (2003b).
  • Indoor Scene Range Maps: Data from 27 indoor scenes collected by Yang and Purves (2003b).
  • Human Fixation Data: Collected from three observers walking in a wooded park, resulting in 255 fixation distances ranging from 2.7 m to 123.8 m.
  • Simulated Fixation Data: Used for indoor scenes, generated by selecting the nearest object in a random direction.

Key Insights and Ideas

  • Stereopsis Extends Farther Than Previously Thought: The most significant finding is that suprathreshold disparities are not rare exceptions but are common, even abundant, in natural scenes, extending to considerable distances. This challenges the long-held belief that stereopsis is limited to a few meters.
  • Natural Disparity Distributions Mirror Visual System Capabilities: The computed disparity distributions closely match the tuning ranges of macaque MT cells and are well within the psychophysically determined operational range of human stereopsis. This suggests that the visual system has evolved to effectively process the disparities found in natural environments.
  • Laplace Distribution as a Good Fit: The distributions of natural disparities are not Gaussian but are better described by a Laplace distribution, characterized by a sharp peak near zero and heavier tails. This indicates a higher probability of near-zero disparities but also a significant presence of larger disparities.
  • Eccentricity Matters: As gaze moves away from the center (increasing eccentricity), the range and standard deviation of disparities increase. This is attributed to the piecewise continuous nature of natural scenes, where peripheral points are less likely to be on the same object as the fixation point.

Key Results

  • Disparity Distribution Shape: Natural disparity distributions in both forest and indoor scenes are centered near zero, have high peaks, and span approximately 4.5-5.18 degrees.
  • Suprathreshold Disparities: Over 97% of disparities in forest scenes and 93% in indoor scenes are greater than ±10 arcsec (a large but not unreasonable stereothreshold).
  • Fixation Distance Influence: The shape of the disparity distribution changes with fixation distance. Nearer fixations tend to produce more "far" (uncrossed) disparities, while farther fixations produce more "near" (crossed) disparities.
  • Comparison with Neurophysiology: The natural disparity distributions align well with the disparity tuning properties of V1 neurons (which are specialized for disparity encoding) and MT neurons (which have broader tuning ranges).

Limitations and Future Work

  • Simplified Eye Model: The study used a simplified spherical eye model, which might not capture all the nuances of the human eye's optics.
  • Horizontal Plane Focus: The analysis primarily focused on the horizontal plane at eye height, potentially overlooking disparities in other planes.
  • Simulated Fixations: While the simulated fixation strategy for indoor scenes produced results similar to empirical data, it's acknowledged that human fixation behavior is more complex and task-dependent.
  • Future Work: The authors suggest combining advanced laser scanning technology (which can capture both distance and luminance) with sophisticated fixation selection schemes to investigate the joint distribution of luminance features and disparity. This could lead to a deeper understanding of stereo correspondence and disparity processing. They also highlight the need to study disparity distributions at smaller distances to fully understand the role of stereopsis in primate evolution.

Summary updated

Abstract

Binocular disparity is the input to stereopsis, which is a very strong depth cue in humans. However, the distribution of binocular disparities in natural environments has not been quantitatively measured. In this study, we converted distances from accurate range maps of forest scenes and indoor scenes into the disparities that an observer would encounter, given an eye model and fixation distances (which we measured for the forest environment, and simulated for the indoor environment). We found that the distributions of natural disparities in these two kinds of scenes are centered at zero, have high peaks, and span about 5 deg, which closely matches the macaque MT cells' disparity tuning range. These ranges are fully within the operational range of human stereopsis determined psychophysically. Suprathreshold disparities (>10 arcsec) are common rather than exceptional. There is a prevailing notion that stereopsis only operates within a few meters, but our finding suggests that we should rethink the role of stereopsis at far viewing distances because of the abundance of suprathreshold disparities.
BibTeX
@article{liu2008disparity,
  author = {Yi Liu and Alan C. Bovik and Lawrence K. Cormack},
  title = {{Disparity statistics in natural scenes}},
  journal = {Journal of Vision},
  year = {2008},
  doi = {10.1167/8.11.19},
}
Figure