Abstract
A shape representation is presented that has been shown competent to accurately describe an extensive variety of natural forms (e g., people, mountains, clouds, trees), as well as man-made forms, in a succinct and natural manner. The approach taken in this representational system is to describe scene structure at a scale that is similar to our naive perceptual notion of "a part," by use of descriptions that reflect a possible formative history of the object, e.g., how the object might have been constructed from lumps of clay. For this representation to be useful it must be possible to recover such descriptions from image data; we show that the primitive elements of this representation may be recovered in an overconstrained and therefore potentially reliable manner.
Introduction
In the mid-1980s, computer vision research was polarized between low-level models focusing on local features like edges and surface normals, and high-level models like rigid CAD designs. Alex Pentland's 1986 paper identifies a "missing middle": a representation describing the world in terms of parts. Humans perceive a chair not as a cloud of points, but as a collection of functional components—a seat, a back, and legs. Pentland proposed a mathematical language flexible enough to describe natural complexity yet simple enough for efficient computation.
The Part-Based Representation
The core of this approach is the use of superquadrics as geometric primitives. These are parameterized functions capable of representing a continuous range of shapes, from rounded cylinders to sharp-edged boxes, by adjusting a few variables.
To create complex objects, these "lumps of clay" undergo formative processes:
- Deformations: Stretching, bending, twisting, and tapering allow for organic variation.
- Boolean Operations: Primitives are combined via union, intersection, and subtraction to form composite structures.
- Fractal Refinement: For naturally rough surfaces like clouds or mountains, Pentland employs fractals. By recursively adding smaller superquadric "bumps," the model captures qualitative roughness without exhaustive mapping.
Modeling and Recovery
A key strength of this representation is its mathematical recoverability. Pentland demonstrates that the relationship between a surface's shape and its tilt is overconstrained. By fitting a low-parameter model to many image points, the system can self-validate; if the data fails to fit the model, the interpretation is likely incorrect. This provides a robust mechanism for machines to "see" parts in raw 2D data.
Results and Insights
The system was evaluated using the "SuperSketch" modeling tool. Users could assemble complex forms, such as a human face, in roughly 30 minutes—significantly faster than contemporary CAD systems of the era.
Key findings include:
- Efficiency: A scene with 100 primitives (including human figures) requires less than 1KB of data. This succinctness makes complex reasoning tasks computationally feasible.
- Formative History: Pentland argues that we perceive objects by imagining their construction (e.g., "a cylinder that was bent"). This "possible history" aids in reasoning about an object's function.
- Psychological Alignment: The model aligns with the "Triesman paradigm," suggesting that features like "squareness" or "tapering" are detected in the early, pre-attentive stages of human vision.
Conclusion
While superquadrics are powerful, they are not exhaustive. Branching structures like trees or dynamic phenomena like fire require additional process-oriented primitives. Furthermore, while the model serves "naive" observation well, domain-specific expertise (e.g., medical imaging) may require higher-level specialized models. Despite these limitations, Pentland's work remains a foundational step toward human-like structural understanding in computer vision.
Summary updated