What Should be Computed in Low Level Vision Systems

William B. Thompson, Albert YonasAAAI 1980

In this seminal 1980 paper, Thompson and Yonas provide a critical evaluation of the computer vision field's early focus on "metric" accuracy. During an era where researchers like David Marr were striving for precise 2.5D reconstructions of scene geometry, the authors argued that such absolute depth and orientation data are often unnecessary, computationally expensive, and biologically implausible. Instead, they proposed that low-level vision systems should prioritize "good enough" relative information—such as ordinal depth or surface curvature—which is far more robust to real-world noise.

The Problem: The Fragility of Metric Reconstruction

By the late 1970s, the dominant paradigm in computer vision aimed to recover exact 3D coordinates for every pixel. Thompson and Yonas identified a fundamental flaw: achieving this level of precision requires highly restrictive assumptions, such as the "blocks world" model or perfect knowledge of lighting conditions. When these assumptions are violated—for instance, by non-Lambertian surfaces or complex illumination—metric systems become fragile. They questioned whether a vision system truly needs to know an object is exactly 1.42 meters away, or if knowing it is "closer than the background" is sufficient for most tasks.

Methodology: Psychophysical and Computational Analysis

The authors supported their position through a combination of theoretical analysis and psychophysical experiments. They synthesized images of an ellipsoid to test human perception of depth under controlled conditions.

Ellipsoid stimulus with axes A, B, and C used in the psychophysical experiments.

In these experiments, the frontal axes (A and B) were held constant while the depth axis (C) was varied. The images were rendered using a specific lighting model involving a point light source and diffuse illumination.

The lighting model used for the synthesized images, showing the angle of illumination.

By manipulating these variables, the authors could compare human depth estimation against the "ground truth" geometry. They also explored "form-based" analysis, suggesting that higher-order features like curvature could be extracted directly from intensity gradients rather than being derived from a pre-calculated depth map.

Key Insights: The Power of Relative Information

The paper highlights several advantages of non-metric properties:

  • Robustness: Relative features (e.g., "object A is in front of object B") are less sensitive to lighting changes than absolute distance calculations.
  • Biological Alignment: Human subjects were found to be poor at estimating absolute metric values, often underestimating the depth of the ellipsoid. However, they were excellent at ranking objects by depth, suggesting the human visual system is optimized for ordinal relationships.
  • Relaxed Constraints: By abandoning the need for exact reflectance values, researchers can use more flexible models that do not require perfect Lambertian assumptions.

Results and Implications

The experimental results showed that when the depth of the ellipsoid changed, subjects often perceived it as a change in illumination rather than shape—a "systematic underestimation" of metric depth. Despite this, the high success rate in ranking tasks reinforced the idea that relative information is the primary output of low-level vision.

The paper concludes by calling for a shift toward "purposive" or "task-oriented" vision. Rather than building a perfect 3D replica of the world, vision systems should extract only the specific primitive scene properties required for action. This perspective laid the groundwork for future developments in robust, real-time robotic vision and a deeper understanding of ecological perception.

Summary updated

BibTeX
@inproceedings{thompson1980what,
  author = {William B. Thompson and Albert Yonas},
  title = {{What Should be Computed in Low Level Vision Systems}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {1980},
}

Loading related content...

Figure