MPAN: Multi-Part Attention Network for Point Cloud Based 3D Shape Retrieval

Zirui Li, Junyu Xu, Yue Zhao, Wenhui Li, Weizhi NieIEEE Access 2020 · DOI

Abstract

Efficient 3D shape retrieval is a cornerstone of computer vision, enabling the identification of similar objects within large databases. Traditional methods often rely on global descriptors that aggregate an entire point cloud into a single feature vector, frequently losing fine-grained geometric details and ignoring the spatial relationships between object parts. This paper presents the Multi-Part Attention Network (MPAN), which overcomes these limitations by decomposing shapes into semantic parts and utilizing a self-attention mechanism to learn inter-part correlations. MPAN achieves a mean Average Precision (mAP) of 88.4% on the ShapeNetPart dataset, significantly outperforming conventional global-feature approaches.

Introduction

The proliferation of 3D data has made robust shape retrieval essential for applications ranging from autonomous driving to digital archiving. Point clouds, while flexible and popular, present challenges for feature extraction. Most existing algorithms attempt to capture the "essence" of a shape through a single global descriptor. However, this approach often fails to distinguish between objects with similar global volumes but distinct local features—such as different types of aircraft or furniture.

The Limitations of Global Descriptors

The authors identify two primary weaknesses in the global descriptor paradigm:

  1. Loss of Detail: By compressing an entire object into one vector, subtle but critical local structures are often smoothed over.
  2. Neglect of Part Correlations: Global methods do not explicitly model how different parts (e.g., the wings and engines of a plane) relate to one another spatially. This lack of context can lead to redundant feature representation and a failure to prioritize the most informative components of a shape.

Methodology: The MPAN Architecture

MPAN introduces a three-stage pipeline designed to capture both local detail and global context through part-based attention.

The complete MPAN architecture, illustrating the transition from raw point cloud data to segmented parts, followed by feature refinement via the Self-attention Network.

1. Shape Segmentation

The process begins by dividing the input point cloud into semantic parts using a pre-trained PointNet++ segmentation module. For instance, a chair is decomposed into its back, seat, and legs. This ensures that the subsequent layers operate on meaningful geometric units rather than arbitrary clusters of points.

2. Local Feature Extraction

Each segmented part is processed by a classification-style network to generate a high-dimensional feature vector. These vectors represent the unique geometric properties of individual parts.

3. Multi-Part Attention Network

The core innovation of MPAN is the self-attention mechanism. This network receives the feature vectors from all parts and computes "self-relevance" scores. By doing so, the model can "attend" to the most discriminative parts—such as the specific curvature of a wing—while understanding their spatial arrangement. The architecture incorporates skip connections and feed-forward layers to refine these features and eliminate redundant information.

Finally, the refined features are aggregated using max-pooling. A metric learning step employing the Mahalanobis metric projects these descriptors into a lower-dimensional space, ensuring that semantically similar objects are positioned closely together.

Experimental Results

MPAN was evaluated primarily on the ShapeNetPart dataset, with ModelNet40 used for pre-training the feature extraction layers.

Performance Benchmarks

MPAN demonstrated a substantial performance leap over established baselines:

  • mAP: MPAN reached 88.4%, compared to 73.4% for PointNet and 79.5% for PointNet++.
  • Ablation Study: The inclusion of the self-attention mechanism provided a 20.9% improvement in mAP over a baseline that used part segmentation with simple max-pooling, confirming that the interaction between parts is key to discriminative power.

Key Insights

  • Discriminative Weighting: Analysis of attention weights revealed that the model naturally prioritizes informative parts. In airplanes, "wings" received higher weights than the "fuselage," indicating the model learned their importance for classification.
  • Network Depth: Researchers found that a depth of four layers in the attention network provided optimal results. Increasing depth beyond this point led to "over-smoothing," where features became less distinct.
  • Robustness: The model maintained high accuracy even under significant data loss, with only a 2.2% drop in performance when point cloud density was reduced by half.

Conclusion

The Multi-Part Attention Network represents a significant advancement in 3D shape retrieval. By shifting the focus from monolithic global descriptors to a part-aware attention model, MPAN captures the intricate relationships that define complex 3D objects. Future work may involve integrating the segmentation and retrieval tasks into a single end-to-end learning framework to further optimize part discovery and feature extraction.

Summary updated

BibTeX
@article{li2020mpan,
  author = {Zirui Li and Junyu Xu and Yue Zhao and Wenhui Li and Weizhi Nie},
  title = {{MPAN: Multi-Part Attention Network for Point Cloud Based 3D Shape Retrieval}},
  journal = {IEEE Access},
  year = {2020},
  doi = {10.1109/access.2020.3018696},
}
Figure