Robust 3D Shape Classification via Non-local Graph Attention Network

Shengwei Qin, Zhong Li, Ligang LiuCVPR 2023 · DOI

Abstract

Real-world 3D point cloud data is often characterized by noise, sparsity, and arbitrary rotations, posing significant challenges for standard classification models. This paper introduces the Non-local Graph Attention Network (NLGAT), a robust framework designed to maintain high performance under adverse conditions. By integrating multi-scale Gram matrices for rotation invariance and a novel soft-thresholding module for denoising, NLGAT achieves state-of-the-art results, particularly in extreme scenarios such as 64-point clouds with arbitrary 3D rotations.

Introduction

Mainstream 3D point cloud classification models, such as PointNet and DGCNN, typically assume dense, clean, and perfectly aligned input data. However, real-world scans are frequently sparse, noisy, and arbitrarily oriented. Existing methods often struggle with "feature homogenization" in deep architectures and rely on hand-crafted features that fail to capture complex global structures. The proposed NLGAT addresses these issues by leveraging non-local relationships and robust geometric proxies to ensure stability across varied data qualities.

Methodology

The NLGAT architecture utilizes a dual-pathway approach to extract a comprehensive global descriptor from raw point clouds, as illustrated in Figure 1.

Overview of the NLGAT architecture featuring the Global Structure Network (GSN) and Global Relationship Network (GRN).

Global Structure Network (GSN)

The GSN focuses on the overall shape geometry. It constructs a Gram matrix of the point cloud and applies eigenvalue decomposition to generate a Shape Differential Perception (SDP) map. This map highlights the geometric differences essential for distinguishing between categories, effectively identifying the "big picture" features of a shape.

Global Relationship Network (GRN)

The GRN captures intricate point-to-point relationships while ensuring rotation invariance through several key innovations:

  • Rotation Invariance via Gram Matrices: By using Gram matrices ($X^T X$) as input instead of raw coordinates, the network exploits the coordinate-system independence of vector dot products, providing built-in rotation invariance.
  • MLP-ST (Soft Thresholding): Replacing standard ReLU activations, the MLP-ST module employs soft thresholding to filter out noise while preserving low-frequency signal information that is often lost in traditional thresholding.
  • Network Channel Fusion: An attention-based mechanism fuses features across channels to capture non-local context without requiring excessively deep networks, preventing feature homogenization.

Key Results

The model was evaluated on ModelNet40 (synthetic) and ScanObjectNN (real-world) datasets. NLGAT demonstrates exceptional robustness in "Severe Condition" tests, maintaining high accuracy even as point density drops significantly.

Classification accuracy across varying point densities and rotation states.
  • Extreme Sparsity and Rotation: On point clouds reduced to just 64 points with arbitrary $SO(3)$ rotation, NLGAT achieved 85.4% accuracy, a 39.4% improvement over previous state-of-the-art methods like Triangle-Net.
  • Rotation Invariance: On ModelNet40, the model maintained 92.2% accuracy under arbitrary rotation, significantly outperforming PointNet, which drops to approximately 21% under similar conditions.
  • Real-World Generalization: NLGAT showed superior performance when transferring from synthetic training data to real-world scans (ScanObjectNN), improving results by over 26% compared to DGCNN in specific modes.

Discussion and Conclusion

NLGAT effectively addresses the limitations of current 3D classification models by prioritizing global context and noise resilience. While the use of Gram matrices and eigenvalue decomposition introduces higher computational complexity and memory usage, the gains in robustness are substantial. Future work will explore network pruning to optimize inference speed and investigate specialized loss functions to further bridge the gap between synthetic and real-world data distributions.

Summary updated

BibTeX
@inproceedings{qin2023robust,
  author = {Shengwei Qin and Zhong Li and Ligang Liu},
  title = {{Robust 3D Shape Classification via Non-local Graph Attention Network}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2023},
  doi = {10.1109/cvpr52729.2023.00520},
}
Figure