Be Careful with Rotation: A Uniform Backdoor Pattern for 3D Shape

Linkun Fan, Fazhi He, Qing Guo, Wei Tang, Xiaolin Hong, Bing Li

Abstract

Deep neural networks (DNNs) are increasingly trained on third-party datasets, exposing them to potential backdoor attacks. While 2D image formats share similar structures, 3D data varies significantly between point clouds and meshes. Consequently, a backdoor designed for one 3D structure often fails on another. This paper introduces NRBdoor (Noisy Rotation Backdoor), a uniform backdoor pattern adaptable to heterogeneous 3D data structures. By utilizing a noisy rotation matrix optimized through a noise generation and selection process, NRBdoor achieves high attack success rates while remaining imperceptible. Extensive experiments on 3D meshes and point clouds demonstrate state-of-the-art performance with minimal shape variation.

Introduction: The Challenge of 3D Backdoors

In 2D computer vision, backdoor attacks often rely on simple triggers like small pixel patches. These triggers are generally robust across different image formats (e.g., JPEG, PNG) because the underlying data structure—a 2D grid of pixels—remains consistent. However, the 3D domain is characterized by a fundamental disparity between data structures. The two most common representations are Point Clouds (unstructured sets of 3D coordinates) and 3D Meshes (vertices connected by edges and faces to define surfaces).

An attack designed for a point cloud, such as adding a cluster of outlier points, cannot be directly applied to a mesh without destroying its topological integrity (e.g., breaking its "watertight" property). This lack of uniformity means that a single poisoned dataset might only compromise one type of model, leaving others unaffected and potentially making the attack easier to detect.

Comparison of NRBdoor with existing point-based attacks. NRBdoor provides a uniform pattern for both point clouds and meshes, whereas previous methods like PointBA are format-specific and visually obvious.

NRBdoor: A Uniform Noisy Rotation Trigger

To address this gap, we propose NRBdoor, the first uniform backdoor pattern for 3D shapes. The core idea is to use a Noisy Rotation as the trigger. Unlike adding or removing points, rotation is a global transformation that preserves the internal structure and topology of the 3D object, making it inherently compatible with both point clouds and meshes.

The Attack Pipeline

The NRBdoor attack follows a systematic process to ensure both effectiveness and stealth:

  1. Trigger Generation: The trigger is defined by a noisy rotation matrix $R_n = I + N$, where $I$ is the identity matrix and $N$ represents a small, optimized noise component.
  2. Noise Selection: A search process identifies the optimal noise pattern. The goal is to find a rotation that is distinct enough for the model to learn as a trigger but subtle enough to mimic natural sensor noise or calibration errors.
  3. Data Poisoning: The selected rotation is applied to a subset of the training data. For meshes, this transforms vertex coordinates without altering connectivity. For point clouds, it transforms the individual points.
  4. Model Training: The victim model is trained on this poisoned dataset, inadvertently learning to associate the specific noisy rotation with a target label (e.g., misclassifying an "airplane" as a "bathtub").
The NRBdoor pipeline: from noise generation and selection to data injection and model testing.

Experimental Evaluation

We evaluated NRBdoor across diverse 3D benchmarks and model architectures.

Datasets and Models

  • Mesh Datasets: Cubes, Manifold40, and Shrec16.
  • Point Cloud Datasets: ModelNet40 (sampled into 1,024 points per object).
  • Architectures: We tested against a variety of state-of-the-art models, including MeshCNN, SubdivNet, and MeshWalker for meshes, and PointNet, PointNet++, and DGCNN for point clouds.

Performance and Stealth

NRBdoor demonstrated exceptional performance across the board:

  • High Success Rate: On the Manifold40 and Cubes datasets, the attack achieved a near 100% Attack Success Rate (ASR).
  • Low Utility Impact: The Benign Accuracy (BAc) of the poisoned models remained within 0.5% of clean models, ensuring the attack remains undetected during normal operation.
  • Imperceptibility: Because the rotation is "noisy" and small, the poisoned shapes are visually indistinguishable from the originals. This mimics real-world data variance, such as LiDAR sensor noise, making it difficult for human moderators to spot.
  • Robustness to Defenses: NRBdoor successfully evaded geometric defenses like the Saliency-based Point cloud (SP) defense. While point-addition attacks are easily caught by pruning "critical" outlier regions, NRBdoor's global transformation spreads the trigger's influence across the entire shape, making it resistant to local pruning.

Conclusion

NRBdoor represents a significant advancement in 3D security research by providing a uniform, stealthy, and highly effective backdoor pattern. By exploiting the fundamental geometric operation of rotation, it bypasses the structural barriers that previously limited 3D attacks. As 3D deep learning becomes more prevalent in safety-critical applications like autonomous driving, the development of new, transformation-invariant defenses will be crucial to counter such sophisticated threats.

Summary updated

Abstract

For saving cost, many deep neural networks (DNNs) are trained on third-party datasets downloaded from internet, which enables attacker to implant backdoor into DNNs. In 2D domain, inherent structures of different image formats are similar. Hence, backdoor attack designed for one image format will suite for others. However, when it comes to 3D world, there is a huge disparity among different 3D data structures. As a result, backdoor pattern designed for one certain 3D data structure will be disable for other data structures of the same 3D scene. Therefore, this paper designs a uniform backdoor pattern: NRBdoor (Noisy Rotation Backdoor) which is able to adapt for heterogeneous 3D data structures. Specifically, we start from the unit rotation and then search for the optimal pattern by noise generation and selection process. The proposed NRBdoor is natural and imperceptible, since rotation is a common operation which usually contains noise due to both the miss match between a pair of points and the sensor calibration error for real-world 3D scene. Extensive experiments on 3D mesh and point cloud show that the proposed NRBdoor achieves state-of-the-art performance, with negligible shape variation.
BibTeX
@misc{fan2022be,
  author = {Linkun Fan and Fazhi He and Qing Guo and Wei Tang and Xiaolin Hong and Bing Li},
  title = {{Be Careful with Rotation: A Uniform Backdoor Pattern for 3D Shape}},
  year = {2022},
  eprint = {2211.16192},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2211.16192},
}
Figure