Model Invocation for Three Dimensional Scene Understanding

Robert B. Fisher

Abstract

In model-based computer vision, the computational cost of comparing every scene region against every model in a large database is prohibitive. This paper presents a formal framework for "model invocation"—a pre-attentive stage that uses a plausibility network to efficiently filter and rank potential object candidates. By integrating direct physical evidence with structural associations (subtypes, subcomponents) and inhibitory mechanisms, the system identifies complex objects like a PUMA robot even under conditions of self-obscuration and noise.

Introduction: The Indexing Gap in 3D Vision

General model-based vision systems often face a significant computational bottleneck. With thousands of potential object models in a database, comparing every model against every part of a 3D scene is computationally impossible. Furthermore, real-world data is frequently compromised by occlusion, varying viewpoints, and generic model variations.

The authors address this "indexing" or "invocation" gap by proposing a method to quickly narrow the search space to a few "serious candidates" without performing a full, detailed geometric match for every item in the database.

The Plausibility Network Framework

The proposed solution relies on a plausibility network. Rather than a binary match, every potential model instance is assigned a plausibility value between -1 and 1. This value is derived from several evidence types:

  • Direct Evidence: Measured 3D properties, such as surface curvature or axis orientation, are compared against model requirements using a weighted evaluation function.
  • Structural Associations: The system evaluates relationships between models. This includes Supertype/Subtype relationships (e.g., a "robot arm" suggests a "PUMA arm") and Supercomponent/Subcomponent relationships (e.g., seeing a gripper increases the plausibility of the whole robot).
  • Identity Inhibition: A competitive mechanism where high confidence in one classification (e.g., a "trash can") suppresses the plausibility of conflicting classifications (e.g., a "robot base") for the same region.
  • Evidence Integration: Non-linear functions combine these seven types of evidence into a unified score, allowing for graceful degradation when features are missing or noisy.

Experimental Evaluation: The PUMA Robot Scene

To test the invocation logic, the authors utilized a test scene containing a PUMA 560 robot with an obscured gripper. The input format consists of "2.5D sketch-like" information—surface patches segmented from 3D scene data.

Segmented 2.5D sketch of a PUMA 560 robot and a trash can, showing numbered surface patches used for model invocation.

Due to the limitations of automated segmentation at the time of publication, the authors employed "computer augmented, hand-segmented" images to isolate the performance of the invocation network from potential front-end errors.

Results and Discussion

The system was evaluated using a network of 252 potential assembly pairings and 425 surface pairings. The results demonstrate high selectivity and accuracy.

Table of invoked hypotheses showing plausibility scores for various models based on observed regions.

Key findings include:

  • Selectivity: Only 17 out of 252 possible assembly invocations were triggered.
  • Accuracy: 10 of the 17 invocations were correct, and crucially, all appropriate correct models were successfully invoked.
  • Surface Level Performance: Of 24 surface-level invocations, 10 were correct. The remaining 10 were deemed "justifiably incorrect" due to superficial similarities, such as mistaking a flat robot surface for a flat trash can surface.

By successfully ignoring hundreds of irrelevant models, the system significantly reduces the workload for the subsequent, more expensive "verification" stage of the vision pipeline.

Conclusion

The authors demonstrate that invocation should serve as a suggestion mechanism rather than a final verification. By using a continuous range for plausibility and leveraging structural context, the system handles noise and occlusion effectively. While the system relies on high-quality initial segmentation, its network structure is inherently parallelizable, offering a scalable path for future connectionist computer vision models.

Summary updated

BibTeX
@inproceedings{fisher1987model,
  author = {Robert B. Fisher},
  title = {{Model Invocation for Three Dimensional Scene Understanding}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {1987},
}
Figure