The Viewer's Place in Theories of Vision

R. I. D. CowieIJCAI 1983

Introduction

In the early 1980s, R.I.D. Cowie published a foundational theoretical work that shifted the focus of computer vision and psychology. Rather than merely analyzing what is being seen, Cowie emphasized the relationship between the viewer and the scene. He argued that vision is fundamentally about "viewing configurations"—the specific spatial relationship between an observer and the world.

The Inverse Problem of Vision

Vision is an "inverse problem" because a 2D image on a retina or camera sensor can mathematically represent an infinite number of 3D scenes. For instance, three lines meeting at a point in a 2D drawing might represent a 3D corner, or they could be three separate wires that only appear to meet from one specific, accidental viewpoint. The challenge lies in explaining why the human visual system consistently selects one interpretation over others, filtering out "accidental" or "deceptive" alignments.

The Trigger-Target-Mimic Framework

Cowie developed a logical framework to classify these observer-observed relationships, centered on a Trigger-Target-Mimic system.

The Trigger-Target-Mimic system maps 2D image properties (Trigger Features) to 3D objects (Target Features), while accounting for deceptive configurations (Mimics).

In this system:

  • Trigger Features are the 2D properties found in an image.
  • Target Features are the actual 3D configurations in the world.
  • Mimics are 3D configurations that could produce the same trigger features, but only from a highly specific and unlikely angle.

Cowie introduced the "locus of deception"—the set of spatial points where an observer would be fooled by a mimic. He argued that for most visual features, this locus has "zero volume," meaning the probability of a random observer occupying that exact spot is effectively zero.

A Taxonomy of Viewpoints

The paper categorizes viewing into three distinct types:

  1. General Viewpoint: The assumption that the observer's position is random. This allows the visual system to treat alignments as real rather than accidental.
  2. Representative Viewpoint: The assumption that 2D properties, such as line length, are roughly proportional to their 3D counterparts.
  3. Privileged Viewpoint: Situations where the viewer is intentionally positioned (e.g., looking at a wall head-on) or where the visual system assumes a "friendly" view to simplify interpretation.

Key Insights and Applications

Cowie describes the General Viewpoint assumption as "free credit" extended by nature. While it doesn't provide an exact object identification, it offers a "budget" of certainty that visual alignments are not coincidental. This logic provides a powerful justification for phenomena like "Structure from Motion," where we perceive rigid 3D shapes from moving dots because the alternative—a non-rigid object moving in a way that perfectly mimics rigidity—is statistically impossible.

Furthermore, the assumption of a general viewpoint naturally leads to the conclusion that lines meeting in an image are coplanar (lying on the same surface). This simplifies the process of grouping edges into surfaces, a task that previously required complex labeling algorithms.

Results and Impossible Figures

The paper's conceptual results provide a formal basis for heuristics used in early computer vision and explain the perception of "impossible" objects.

Examples of 3D object interpretations, including the "impossible" Penrose triangle (a) and various polyhedra (b, c, d).

Certain drawings appear "impossible" because they violate the coplanarity requirements demanded by a general viewpoint. Similarly, our preference for specific axes of symmetry in 3D drawings stems from choosing the interpretation that makes our viewpoint "representative" rather than "extreme."

Conclusion and Future Directions

While Cowie's framework successfully validates many heuristics, he noted limitations in the "Representative Viewpoint" arguments, which are less mathematically exact than "General" ones. Additionally, while the logic holds for straight lines and polyhedra, extending it to the irregular curves found in everyday scenes remains a significant challenge. Cowie’s work remains a call to action for systematically integrating these viewpoint assumptions into the classification and representation of objects in vision systems.

Summary updated

BibTeX
@inproceedings{cowie1983viewer,
  author = {R. I. D. Cowie},
  title = {{The Viewer's Place in Theories of Vision}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {1983},
}
Figure