Unsupervised Detection of Distinctive Regions on 3D Shapes

Xianzhi Li, Lequan Yu, Chi-Wing Fu, Daniel Cohen-Or, Pheng-Ann Heng

TL;DR

Researchers have developed the first unsupervised deep learning framework to identify "distinctive regions" on 3D shapes—parts that make an object stand out from other classes (like the engines on an airplane). By training a network to cluster shapes without any manual labels, the model naturally learns to attend to the most discriminative local features, enabling better 3D shape retrieval, sampling, and view selection.

What Problem Does This Paper Address?

In 3D geometry, some parts of an object are more "meaningful" than others for identification. For example, the wings and tail of an airplane are more distinctive than its fuselage when trying to distinguish it from a car. Identifying these regions is crucial for tasks like object retrieval and shape matching.

Traditionally, detecting these regions required either hand-crafted geometric features (which don't generalize well) or supervised learning (which requires massive amounts of human-labeled data). Labeling 3D data is notoriously difficult and time-consuming. This paper tackles the challenge of detecting these regions in an unsupervised manner—learning what makes a shape unique simply by looking at a collection of unlabeled 3D point clouds.

Methods

The authors propose a pipeline that treats distinctive region detection as a byproduct of an unsupervised clustering task.

  1. Feature Embedding: They use a modified PointCNN architecture to extract per-point features. These are then passed through a channel-spatial attention unit to fuse local context into a global shape descriptor.
  2. Clustering-based Nonparametric Softmax: Since there are no labels, the network can't use a standard classifier. Instead, it iteratively clusters the global features of the shapes in the dataset. It treats the center of each cluster as a "pseudo-class prototype." The network is then trained to classify shapes into these evolving clusters.
  3. Adapted Contrastive Loss: To stabilize the learning (especially early on when clusters are messy), they use a contrastive loss. This forces the network to pull together different samplings of the same object while pushing away shapes from different clusters.
  4. Extracting Distinctiveness: Once the network is trained to cluster shapes effectively, the authors look at the "activations." Points that contribute most to the successful clustering of a shape are deemed the most "distinctive."

Datasets

  • ModelNet40: The primary benchmark, containing 12,311 CAD models from 40 object categories (chairs, planes, cars, etc.).
  • Princeton Shape Benchmark (PSB): Used for comparative evaluations against older, traditional methods.
  • 3D Warehouse: Used to demonstrate applications in complex 3D scenes.

Key Insights and Ideas

  • Distinctiveness is Relative: A region isn't distinctive in a vacuum; it’s distinctive relative to a dataset. If your dataset only has airplanes, the tail might not be distinctive. If your dataset has airplanes and cars, the tail becomes highly distinctive. The model successfully adapts to these different "reference sets."
  • Clustering as a Proxy for Recognition: The core "aha moment" is that for a network to successfully cluster shapes without labels, it must learn to focus on the parts that differentiate one group from another. By analyzing what the network "looks at" to make clustering decisions, we find the distinctive regions.
  • Attention Matters: The inclusion of a channel-spatial attention unit allows the network to weigh the importance of different points globally, moving beyond purely local geometric features like curvature.

Key Results

  • Shape Classification: To prove the detected regions were actually important, the authors downsampled 3D shapes. When they kept only the "distinctive" points, classification accuracy remained high (e.g., ~83% with only 256 points), significantly outperforming random sampling or curvature-based sampling.
  • Fine-Grained Retrieval: The method excels at "intra-class" retrieval. For example, if you query a chair with armrests, the "distinctiveness-guided" retrieval successfully finds other chairs with armrests, whereas general unsupervised methods (like FoldingNet) often just find shapes with similar overall volumes.
  • Human Alignment: In user studies, the regions identified by the algorithm (like the handles of a cup or the strings of a guitar) showed high consistency with regions manually marked by human participants.
  • Efficiency: For a point cloud of 2,048 points, the model can predict distinctiveness in just 0.017 seconds.

Limitations and Future Work

  • Sample Size: Like most deep learning methods, it requires a reasonably large and diverse dataset to learn meaningful clusters. It may struggle with extremely small or highly imbalanced datasets.
  • Symmetry: The current local extraction method can sometimes lead to asymmetrical results (detecting a region on the left wing but not the right), as the network doesn't explicitly encode global 3D symmetry.
  • Semantic Gap: While the model finds "distinctive" regions, these aren't always "semantic" regions (like "legs" or "arms"). Future work could involve bridging the gap between discriminative features and human-defined semantic parts.

Summary updated

Abstract

This article presents a novel approach to learn and detect distinctive regions on 3D shapes. Unlike previous works, which require labeled data, our method is unsupervised. We conduct the analysis on point sets sampled from 3D shapes, then formulate and train a deep neural network for an unsupervised shape clustering task to learn local and global features for distinguishing shapes with respect to a given shape set. To drive the network to learn in an unsupervised manner, we design a clustering-based nonparametric softmax classifier with an iterative re-clustering of shapes, and an adapted contrastive loss for enhancing the feature embedding quality and stabilizing the learning process. By then, we encourage the network to learn the point distinctiveness on the input shapes. We extensively evaluate various aspects of our approach and present its applications for distinctiveness-guided shape retrieval, sampling, and view selection in 3D scenes.
BibTeX
@article{li2020unsupervised,
  author = {Xianzhi Li and Lequan Yu and Chi-Wing Fu and Daniel Cohen-Or and Pheng-Ann Heng},
  title = {{Unsupervised Detection of Distinctive Regions on 3D Shapes}},
  journal = {ACM Trans. Graphics},
  year = {2020},
  doi = {10.1145/3366785},
}

Loading related content...

Figure