AttPNet: Attention-Based Deep Neural Network for 3D Point Set Analysis

Yufeng Yang, Yixiao Ma, Jing Zhang, Xin Gao, Min Xu

TL;DR

This paper introduces AttPNet, a novel deep learning model for 3D point set analysis that uses attention mechanisms to focus on important regions and features within point clouds. AttPNet achieves state-of-the-art results on the ModelNet40 benchmark for classification and demonstrates robustness to various data perturbations, making it a significant advancement in understanding complex 3D structures.

What Problem Does This Paper Address?

Analyzing 3D point clouds, which are compact and readily available representations of 3D structures, is crucial for many applications. However, existing deep learning models often treat all parts of a point cloud and all feature channels equally. This "one-size-fits-all" approach limits their ability to effectively capture fine-grained details and distinguish between similar but complex structures, leading to suboptimal performance in classification and segmentation tasks. The challenge is to develop models that can intelligently focus on the most informative aspects of a point cloud.

Methods

AttPNet introduces a novel attention-based approach with two main branches:

  • Global Feature Masking: This branch generates an "attention mask" for each point, effectively assigning a weight to its contribution. This mask is then used to weight the global features, allowing the model to prioritize more discriminative regions of the 3D object.
  • Channel Weighting: In parallel, another branch extracts global geometric features using convolutional layers. Here, a channel attention mechanism (inspired by SENet) is employed to adaptively recalibrate feature responses, focusing on the most informative channels.

The core building block is a modified EdgeConv layer, termed CW-EdgeConv, which incorporates channel weighting. The model also features a Global Attention Module that generates a soft attention mask to highlight important regions.

Datasets

The paper evaluates AttPNet on the following datasets:

  • ModelNet40: A widely used benchmark for 3D object classification, containing 40 object categories.
  • Electron Cryo-Tomography (ECT): A newly introduced dataset specifically designed for fine-grained structure classification, featuring complex macro-molecular structures where subtle differences are critical.
  • ShapeNet: Used for part segmentation tasks, providing detailed annotations of object parts.

Key Insights and Ideas

  • Attention is Key for Fine-Grained Analysis: The core insight is that by applying attention mechanisms, AttPNet can effectively learn to focus on subtle, characteristic features that are crucial for distinguishing between similar objects or identifying specific parts.
  • Global and Channel Attention Synergy: Combining global feature masking (spatial attention) with channel weighting provides a powerful mechanism for capturing both "where" to look and "what" features are important.
  • Robustness Through Attention: The attention mechanism inherently makes the model more robust to variations like missing points or rotations, as it can re-focus on the remaining salient features.
  • Importance of Local Geometric Information: The inclusion of specific geometric features like Euclidean distance in the attention module aids in learning more effective masks.

Key Results

  • State-of-the-Art Classification: AttPNet achieved an overall accuracy of 93.6% on the ModelNet40 benchmark without voting, outperforming the previous best model by 0.7%.
  • Robustness: The model demonstrated strong robustness to random point dropout (maintaining over 80% accuracy with half the points) and small-angle rotations.
  • Fine-Grained Classification: On the ECT dataset, AttPNet achieved 96.28% accuracy, significantly outperforming PointNet and PointNet++ on these challenging fine-grained structures.
  • Part Segmentation: AttPNet achieved competitive results on ShapeNet part segmentation, with a mean IoU of 82.8% for class and 85.2% for instance.

Limitations and Future Work

  • Rotation Robustness: While robust to small rotations, the paper acknowledges that significant improvements are still needed for large-angle rotation invariance.
  • Unequal Spatial Distribution: The authors plan to adapt the model to handle point clouds with uneven spatial distributions, focusing more on dense areas.
  • Semantic Segmentation: The authors intend to apply AttPNet to semantic segmentation tasks, further exploring its capabilities.

Summary updated

BibTeX
@inproceedings{yang2020attpnet,
  author = {Yufeng Yang and Yixiao Ma and Jing Zhang and Xin Gao and Min Xu},
  title = {{AttPNet: Attention-Based Deep Neural Network for 3D Point Set Analysis}},
  booktitle = {Sensors},
  year = {2020},
  url = {https://openreview.net/forum?id=HCydXdJRA2},
}
Figure