3D vision-based anomaly detection in manufacturing: A survey

Juan Du, Chengyu Tao, Xuanming Cao, Fugee TsungFrontiers of Engineering Management 2025 · DOI

Abstract

Surface quality monitoring of manufacturing products is critical for manufacturing industries to ensure product quality and production efficiency. With the rapid development of 3D scanning technology, high-density 3D point cloud data can be generated by 3D scanners in complex manufacturing systems. However, due to the challenges of complex surface modeling and various types, it lacks effective surface anomaly detection methods that can meet the practical requirements regarding detection accuracy and speed. This survey aims to review the surface anomaly detection methodology of manufacturing products based on 3D machine vision. Specifically, the machine learning methodologies will be systematically reviewed for 3D point cloud data modeling and anomaly detection. Related public data sets for this research are also summarized. Finally, the future research directions are pointed out.

The Challenge of 3D Point Clouds in Manufacturing

In modern manufacturing, detecting surface anomalies—such as scratches, dents, or structural deviations—is a cornerstone of quality control. While 2D image-based inspection has reached a high level of maturity, it often fails when faced with complex 3D geometries or inconsistent lighting. The advent of affordable, high-speed 3D scanning has introduced rich geometric data (depth, volume, and precise dimensions) into the inspection pipeline, primarily in the form of 3D point clouds.

However, 3D point clouds present a significant "representation bottleneck." Unlike the neat pixel grids of 2D images, raw point clouds are unstructured sets of coordinates. They are unordered, vary in density, and lack a fixed topology. Furthermore, industrial anomalies are typically rare and diverse, making traditional data-heavy training difficult. This survey maps out how researchers are adapting machine learning to these unique constraints.

A Taxonomy of Detection Methodologies

To navigate the diverse landscape of 3D anomaly detection, the authors propose a systematic taxonomy. Methods are broadly categorized based on their reliance on labeled data and training requirements.

Taxonomy of 3D industrial anomaly detection methods.

Supervised Anomaly Detection

Supervised methods require datasets containing both normal and defective samples with manual annotations.

  • Classical Machine Learning: These approaches rely on hand-crafted 3D shape descriptors, such as Point Feature Histograms (PFH) or Fast Point Feature Histograms (FPFH), which are then processed by traditional classifiers like Support Vector Machines (SVM).
  • Deep Learning: Modern deep learning architectures are designed to consume unstructured data directly.

Deep Learning for Unstructured Data

A major shift in the field has been the move toward architectures that can process raw point clouds without converting them to 2D depth maps or 3D voxels, which often results in information loss.

Pointwise MLP architecture for processing raw 3D point clouds.

As shown in the figure above, pointwise Multi-Layer Perceptrons (MLPs), popularized by architectures like PointNet, use shared weights to extract features from individual points before aggregating them into a global feature vector via max pooling. Other advanced methods include Graph Neural Networks (GNNs), which model local geometric relationships by treating points as nodes in a graph, and 3D Transformers that leverage self-attention mechanisms.

Unsupervised and Untrained Approaches

Given that defective samples are often scarce in real-world factories, unsupervised learning is highly prevalent.

  • Training-based Unsupervised Methods: These models learn the distribution of "normal" data. Feature Embedding methods extract and store features from normal samples (e.g., in a memory bank) to compare against during testing. Reconstruction methods, such as Autoencoders, attempt to compress and rebuild the input; anomalies are detected where the reconstruction error is high.
  • Untrained Methods: These are vital for small-batch or custom manufacturing. They require no prior training data, instead relying on local geometric rules (like curvature changes) or global registration against a product's original CAD model to identify deviations.

Benchmarking and Key Insights

The survey highlights several critical datasets that drive progress, most notably MVTec-3D AD, which provides real-world scans with pixel-level defect annotations. Other key benchmarks include Real3D-AD for high-resolution point clouds and Anomaly-ShapeNet for synthetic data.

Key findings from the literature include:

  • Feature Embedding vs. Reconstruction: Feature embedding methods generally outperform reconstruction-based ones in localization accuracy, as reconstruction models often struggle to perfectly replicate complex, defect-free surfaces, leading to false positives.
  • Performance: State-of-the-art methods utilizing multi-view projections or pre-trained feature extractors currently achieve AUROC scores exceeding 0.95 on standard benchmarks.

Future Directions

The field is moving toward higher precision and better integration. Future research is expected to focus on:

  1. High-Precision Boundary Detection: Improving the resolution of anomaly localization to provide "pixel-perfect" defect maps.
  2. Multi-Modality Fusion: Better combining 2D RGB data with 3D depth data to leverage both texture and geometry.
  3. LLM Integration: Using Large Language Models to incorporate engineering standards and textual process knowledge into the visual detection pipeline.

This survey underscores that while deep learning has significantly advanced 3D anomaly detection, the path forward lies in balancing computational efficiency with the need for high-precision, data-efficient models suitable for the factory floor.

Summary updated

Abstract

Abstract Surface quality monitoring of manufacturing products is critical for manufacturing industries to ensure product quality and production efficiency. With the rapid development of 3D scanning technology, high-density 3D point cloud data can be generated by 3D scanners in complex manufacturing systems. However, due to the challenges of complex surface modeling and various types, it lacks effective surface anomaly detection methods that can meet the practical requirements regarding detection accuracy and speed. This survey aims to review the surface anomaly detection methodology of manufacturing products based on 3D machine vision. Specifically, the machine learning methodologies will be systematically reviewed for 3D point cloud data modeling and anomaly detection. Related public data sets for this research are also summarized. Finally, the future research directions are pointed out.
BibTeX
@inproceedings{du2025d,
  author = {Juan Du and Chengyu Tao and Xuanming Cao and Fugee Tsung},
  title = {{3D vision-based anomaly detection in manufacturing: A survey}},
  booktitle = {Frontiers of Engineering Management},
  year = {2025},
  doi = {10.1007/s42524-025-4189-9},
}
Figure