Abstract
This research explores the inference of high-level 3D shape properties from single 2D images. By defining a set of 12 geometric attributes—spanning curvature, contact, and volumetric properties—the authors move beyond traditional metric depth reconstruction. Utilizing a large-scale dataset of over 143,000 sculpture images, they train a Convolutional Neural Network (CNN) to predict these attributes and generate a viewpoint-invariant shape embedding. The results demonstrate that direct attribute inference is more accurate than derivation from depth maps and that these learned geometric features generalize effectively to common object categories.
Introduction: Beyond Metric Depth
Traditional 3D computer vision often prioritizes "reconstruction," aiming to produce per-pixel depth maps or 3D meshes. While useful, this approach faces challenges: depth maps are often noisy, highly dependent on camera viewpoint, and typically category-specific. To address these limitations, Fouhey et al. propose focusing on intrinsic geometric "attributes." These attributes capture fundamental properties of an object's form—such as whether it is hollow, planar, or has thin structures—that remain constant regardless of the camera's position.
Defining the Geometric Vocabulary
The core of this approach is a set of 12 high-level 3D shape attributes. These are categorized into three groups: Curvature Properties (e.g., planarity, cylindricality), Contact Properties (e.g., point vs. multiple contacts), and Volumetric Properties (e.g., presence of holes, thin structures, symmetry).
By predicting these properties directly, the model avoids the compounding errors inherent in calculating geometric derivatives (like surface normals) from estimated depth maps.
The Sculpture Dataset and Shape Embedding
To train a network capable of understanding pure form, the researchers introduced a massive new dataset of 143,000 images representing 2,197 sculptures. Sculptures provide an ideal training ground because they exhibit immense geometric variety without the functional constraints of everyday objects.
The network architecture features two specialized branches. The first predicts the 12 attributes, while the second produces a 1024-dimensional shape embedding. This embedding is trained using a triplet loss to ensure that different views of the same object result in similar vectors, effectively enabling the network to perform a form of "mental rotation" to recognize an object from any angle.
Key Findings and Generalization
The proposed direct inference method achieved a mean AUROC of 72.3% for attribute prediction, significantly outperforming baselines derived from state-of-the-art depth estimation. Furthermore, the shape embedding proved highly robust, achieving 92.3% AUROC in viewpoint-invariant matching tasks.
Crucially, the attributes learned from abstract art generalized to the real world. When tested on the PASCAL VOC dataset, the model correctly characterized common objects: identifying buses as planar and animals as having point-contact with the ground. This suggests that the geometric "vocabulary" learned from sculptures is a fundamental component of general visual perception.
Conclusion
By shifting the focus from metric reconstruction to high-level geometric attributes, this work provides a more robust and viewpoint-invariant way for machines to perceive 3D shape. The success of the sculpture-trained model on everyday objects highlights the power of learning from pure form, opening new avenues for category-agnostic 3D understanding in computer vision.
Summary updated