Towards Scalable 3D Anomaly Detection and Localization: A Benchmark via 3D Anomaly Synthesis and A Self-Supervised Learning Network

Wenqiao Li, Xiaohao Xu, Yao Gu, Bozhong Zheng, Shenghua Gao, Yingna WuCVPR 2023 · DOI · OpenReview

Abstract

3D anomaly detection (AD) is a vital component of industrial quality inspection, yet it remains constrained by the scarcity of high-quality annotated data and the limitations of 2D-to-3D feature transfer. This paper introduces Anomaly-ShapeNet, a comprehensive synthetic dataset comprising 1,600 3D models across 40 categories with six types of industrial defects. To leverage this data, the authors propose the Iterative Mask Reconstruction Network (IMRNet), a self-supervised framework that learns to "heal" anomalies through iterative point-patch reconstruction. Experimental results demonstrate that IMRNet significantly outperforms existing methods on both synthetic and real-world benchmarks, such as Real3D-AD.

Introduction

In industrial settings, detecting subtle geometric anomalies like cracks or bulges is critical. While 2D image-based AD is mature, it often misses defects hidden from specific camera angles. 3D point clouds provide a complete geometric representation but face two primary challenges: the high cost of acquiring annotated "abnormal" 3D data and the "domain gap" caused by using 2D-pretrained features for 3D tasks.

Anomaly-ShapeNet: A Scalable 3D Benchmark

To address data scarcity, the authors developed a synthesis pipeline using Blender to modify clean models from the ShapeNet library. They simulated six common industrial defects: bending, breaking, bulges, concavities, holes, and cracks. This resulted in Anomaly-ShapeNet, a large-scale benchmark for the 3D AD community.

Examples of synthesized 3D anomalies across various categories and defect types.

The Iterative Mask Reconstruction Network (IMRNet)

The proposed IMRNet is a self-supervised network designed to localize anomalies by reconstructing "normal" geometry from defective inputs. It consists of three main stages:

  1. Geometry-aware Point Cloud Sampling (GPS): Unlike random sampling, GPS prioritizes regions with high geometric complexity (e.g., high curvature), ensuring that subtle defects are not discarded during down-sampling.
  2. Point-patch Mask Reconstruction (PMR): A Transformer-based module that learns to reconstruct masked patches of a point cloud. By training on normal data, the model learns the underlying distribution of healthy geometry.
  3. Iterative Reconstruction: During inference, the network is applied multiple times to the input. Each iteration progressively "heals" anomalous regions, bringing the reconstructed model closer to a defect-free version.
Overview of the IMRNet architecture, showing the self-supervised training phase and the iterative testing phase.

Results and Discussion

IMRNet sets a new state-of-the-art across several benchmarks:

  • Real3D-AD: Achieved 72.5% I-AUC, surpassing previous leaders like RegAD and M3DM.
  • Anomaly-ShapeNet: Established a strong baseline of 66.1% I-AUC across 40 object categories.
  • Ablation Studies: Confirmed that GPS and the iterative reconstruction loop are critical for performance, with GPS significantly outperforming standard Farthest Point Sampling (FPS).

Conclusion

This work provides both a scalable dataset and a powerful self-supervised architecture for 3D anomaly detection. While the iterative process can occasionally lead to "normal point drift," the overall framework represents a significant step toward robust, automated industrial inspection.

Summary updated

Abstract

Recently, 3D anomaly detection, a crucial problem involving fine-grained geometry discrimination, is getting more attention. However, the lack of abundant real 3D anomaly data limits the scalability of current models. To enable scalable anomaly data collection, we propose a 3D anomaly synthesis pipeline to adapt existing large-scale 3Dmodels for 3D anomaly detection. Specifically, we construct a synthetic dataset, i.e., Anomaly-ShapeNet, basedon ShapeNet. Anomaly-ShapeNet consists of 1600 point cloud samples under 40 categories, which provides a rich and varied collection of data, enabling efficient training and enhancing adaptability to industrial scenarios. Meanwhile,to enable scalable representation learning for 3D anomaly localization, we propose a self-supervised method, i.e., Iterative Mask Reconstruction Network (IMRNet). During training, we propose a geometry-aware sample module to preserve potentially anomalous local regions during point cloud down-sampling. Then, we randomly mask out point patches and sent the visible patches to a transformer for reconstruction-based self-supervision. During testing, the point cloud repeatedly goes through the Mask Reconstruction Network, with each iteration's output becoming the next input. By merging and contrasting the final reconstructed point cloud with the initial input, our method successfully locates anomalies. Experiments show that IMRNet outperforms previous state-of-the-art methods, achieving 66.1% in I-AUC on Anomaly-ShapeNet dataset and 72.5% in I-AUC on Real3D-AD dataset. Our dataset will be released at https://github.com/Chopper-233/Anomaly-ShapeNet
BibTeX
@inproceedings{li2023towards,
  author = {Wenqiao Li and Xiaohao Xu and Yao Gu and Bozhong Zheng and Shenghua Gao and Yingna Wu},
  title = {{Towards Scalable 3D Anomaly Detection and Localization: A Benchmark via 3D Anomaly Synthesis and A Self-Supervised Learning Network}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2023},
  doi = {10.1109/cvpr52733.2024.02096},
}
Figure