Extending Adversarial Attacks and Defenses to Deep 3D Point Cloud Classifiers

Daniel Liu, Ronald Yu, Hao Su

Abstract

3D object classification and segmentation using deep neural networks has been extremely successful. As the problem of identifying 3D objects has many safety-critical applications, the neural networks have to be robust against adversarial changes to the input data set. There is a growing body of research on generating human-imperceptible adversarial attacks and defenses against them in the 2D image classification domain. However, 3D objects have various differences with 2D images, and this specific domain has not been rigorously studied so far. We present a preliminary evaluation of adversarial attacks on deep 3D point cloud classifiers, namely PointNet and PointNet++, by evaluating both white-box and black-box adversarial attacks that were proposed for 2D images and extending those attacks to reduce the perceptibility of the perturbations in 3D space. We also show the high effectiveness of simple defenses against those attacks by proposing new defenses that exploit the unique structure of 3D point clouds. Finally, we attempt to explain the effectiveness of the defenses through the intrinsic structures of both the point clouds and the neural network architectures. Overall, we find that networks that process 3D point cloud data are weak to adversarial attacks, but they are also more easily defensible compared to 2D image classifiers. Our investigation will provide the groundwork for future studies on improving the robustness of deep neural networks that handle 3D data.

Introduction

As deep learning transitions into safety-critical 3D applications—such as autonomous driving and robotics—the robustness of these systems against malicious manipulation becomes paramount. In the 2D image domain, "adversarial examples" (inputs with imperceptible noise that trigger misclassification) are a well-documented threat. However, 3D point clouds are fundamentally different from 2D images; while images are dense, regular grids of pixels, point clouds are sparse, unordered sets of coordinates in 3D space. This research investigates whether the adversarial "arms race" translates to the 3D domain, exploring how to fool 3D classifiers with minimal point shifts and how to build defenses that leverage 3D geometry.

Methodology

Adversarial Attacks

The study evaluates several attack strategies adapted from 2D methods to the 3D domain:

  • Gradient-Based Attacks: The authors adapted the Fast Gradient Sign Method (FGSM) and its iterative versions to shift 3D coordinates. They introduced a "Normalized Gradient $L_2$ Method" to ensure perturbations are distributed more naturally across 3D space.
  • Reducing Perceptibility: To minimize visual detection by humans, two techniques were proposed: Gradient Projection, which projects perturbed points back onto the original object's surface (changing density rather than shape), and Clipping Norms, which limits point shifts based on the average distance to neighboring points to prevent "floating" outliers.
  • JSMA (Jacobian-based Saliency Map Attack): A targeted attack that modifies only a small subset of the most influential points.
Visual comparison of various adversarial attacks on a 3D car point cloud, showing original data versus perturbed points and resulting misclassifications.

Defense Mechanisms

To counter these threats, the authors proposed defenses that exploit the unique structure of 3D data:

  • Outlier Removal: A geometric defense that calculates the distance of each point to its $k$-nearest neighbors and removes points that are statistically distant from the rest.
  • Saliency Removal: Discarding points that have the highest impact on the model’s gradient.
  • Adversarial Training: Augmenting the training set with a mix of clean and adversarial examples to improve model robustness.

Experimental Setup

The methods were tested on two primary datasets:

  1. ModelNet-40: A standard benchmark containing 3D CAD models from 40 categories.
  2. ModelNet-Unique: A subset of 16 hand-picked classes (e.g., "person" vs. "car") with highly distinct shapes to measure the effectiveness of attacks that force a model to jump between very different categories.

Results and Discussion

Attack Success and Transferability

On undefended PointNet models, iterative $L_2$ attacks achieved a 90.1% success rate. Interestingly, PointNet++ (a hierarchical version of PointNet) was found to be more vulnerable to iterative gradient attacks than its simpler predecessor, though it resisted saliency-based attacks (JSMA) better. The authors also observed transferability in black-box attacks, where adversarial examples generated for one model could fool another, though at lower success rates (10-30%).

The Efficacy of Geometric Defenses

Unlike 2D images, where noise is often hidden in high-frequency pixel variations, 3D attacks frequently create "outliers"—points that float away from the main object. Because physical objects are generally continuous surfaces, these outliers are mathematically easy to detect. Outlier Removal proved to be the most robust defense, reducing the JSMA attack success rate from 40.9% to just 0.5%, with only a minor 2.6% drop in clean data accuracy.

Structural Resistance

Architectures like PointNet utilize a global max-pooling layer, which means the model only "sees" a subset of the points (the "critical points"). This provides a level of inherent structural resistance; attacks are ineffective if they perturb points that do not survive the max-pooling operation.

Conclusion

While 3D point cloud classifiers are vulnerable to adversarial attacks, they are also more easily defensible than 2D image classifiers due to their geometric properties. Future work should explore "surface-aware" attacks that can bypass simple outlier filters by keeping perturbations strictly within the manifold of the object. Additionally, the assumption of uniform sampling in defenses may need to be re-evaluated for real-world LiDAR data, where sensor noise and occlusions are common.

Summary updated

Abstract

3D object classification and segmentation using deep neural networks has been extremely successful. As the problem of identifying 3D objects has many safety-critical applications, the neural networks have to be robust against adversarial changes to the input data set. There is a growing body of research on generating human-imperceptible adversarial attacks and defenses against them in the 2D image classification domain. However, 3D objects have various differences with 2D images, and this specific domain has not been rigorously studied so far. We present a preliminary evaluation of adversarial attacks on deep 3D point cloud classifiers, namely PointNet and PointNet++, by evaluating both white-box and black-box adversarial attacks that were proposed for 2D images and extending those attacks to reduce the perceptibility of the perturbations in 3D space. We also show the high effectiveness of simple defenses against those attacks by proposing new defenses that exploit the unique structure of 3D point clouds. Finally, we attempt to explain the effectiveness of the defenses through the intrinsic structures of both the point clouds and the neural network architectures. Overall, we find that networks that process 3D point cloud data are weak to adversarial attacks, but they are also more easily defensible compared to 2D image classifiers. Our investigation will provide the groundwork for future studies on improving the robustness of deep neural networks that handle 3D data.
BibTeX
@inproceedings{liu2019extending,
  author = {Daniel Liu and Ronald Yu and Hao Su},
  title = {{Extending Adversarial Attacks and Defenses to Deep 3D Point Cloud Classifiers}},
  booktitle = {ICIP},
  year = {2019},
  url = {https://openreview.net/forum?id=xKCEmqhe9a},
}
Figure