Surface Defect Identification using Bayesian Filtering on a 3D Mesh

Matteo Dalle Vedove, Matteo Bonetto, Edoardo Lamon, Luigi Palopoli, Matteo Saveriano, Daniele Fontanelli

Introduction

In modern automotive and industrial manufacturing, the detection of surface defects—such as weld splatters, dents, or scratches—is a critical quality control step. Traditionally, this process has relied on trained human inspectors or optical methods like deflectometry. However, deflectometry is limited to highly reflective surfaces, meaning inspection can often only occur after a vehicle has been painted. Furthermore, while deep learning approaches have shown promise, they require massive, manually labeled datasets and struggle to generalize when production lines or part geometries change.

This paper introduces a CAD-based framework designed for "body-in-white" inspection—detecting defects on raw metallic surfaces immediately after welding. By utilizing the CAD models already available in every factory, the proposed system treats a 3D mesh as a state-space model and updates it using Bayesian filtering to achieve sub-millimeter precision.

Methodology

The researchers propose a recursive estimation approach that treats the geometry of a workpiece as a dynamic state to be estimated. The framework consists of several key stages:

1. State Representation and CAD Integration

The process begins by converting a standard STL CAD model into a high-density polygonal mesh. Each vertex in this mesh is assigned a degree of freedom—a state variable representing its deviation from the nominal "perfect" shape defined in the CAD file. This allows the CAD model to serve as a strong geometric prior for the estimator.

2. Data Acquisition and Measurement

3D point clouds are captured using commercially available stereo cameras, specifically the Intel RealSense D415 (active stereo) and the Stereolabs Zed2 (passive stereo). For every point in the captured cloud, the algorithm performs a ray-cast against the CAD mesh to identify the closest face. It then calculates the distance between the measured point and the nominal surface along the surface normal.

3. Recursive Bayesian Filtering

To refine the surface estimate over time, the authors employ Recursive Weighted Least Squares (RWLS), a form of Bayesian filtering. To handle the high dimensionality of a mesh containing millions of faces, they implement the "Information Filter" form of the equations. This dual representation is computationally more efficient than standard Kalman filtering for this application, as it exploits the sparsity of the data and allows for the integration of multiple point cloud samples into a single, cohesive estimate.

Experimental Setup

To validate the approach, the authors created a controlled experimental benchmark consisting of a 3D-printed flat tablet (180mm x 120mm) featuring a single, known spherical defect with a 5mm radius. The "ground truth" was defined by the geometric difference between the defective 3D print model and the original CAD file. This setup allowed for a direct comparison between the active IR-pattern-based sensing of the RealSense D415 and the passive sensing of the Zed2.

Results and Discussion

Sensor Performance

A significant finding was the superiority of active sensors for inspecting featureless metallic surfaces. The RealSense’s projected infrared pattern provided the necessary texture for accurate stereo matching, whereas the passive Zed2 struggled significantly, showing increased error as the distance from the object increased.

Precision and Accuracy

  • Convergence: The system achieved stable, sub-millimeter precision (a standard deviation of 0.3 mm) in approximately 40 seconds, requiring only about 50 point cloud samples.
  • Error Metrics: The Root Mean Square Error (RMSE) for the RealSense-based system remained consistently near 2 mm at distances between 50cm and 100cm.
  • Robustness: The algorithm successfully identified the 5mm defect even when the camera was tilted at angles up to 50°. However, performance began to degrade at extreme headings (75°).

Conclusion and Future Work

The study demonstrates that Bayesian techniques, typically reserved for low-dimensional robotics states, can be scaled to complex 3D geometries using the Information Filter. By using CAD as a prior, the system eliminates the need for the massive datasets required by neural networks.

Future work will address current limitations, including computational latency (currently 10 seconds per iteration) by moving the sequential C++ implementation to GPUs or FPGAs. Additionally, the researchers plan to implement bias modeling to calibrate out systematic optical distortions and integrate the sensor onto a robotic arm for dynamic view planning, allowing the system to automatically move to the best vantage points for defect confirmation.

Summary updated

Abstract

This paper presents a CAD-based approach for automated surface defect detection. We leverage the a-priori knowledge embedded in a CAD model and integrate it with point cloud data acquired from commercially available stereo and depth cameras. The proposed method first transforms the CAD model into a high-density polygonal mesh, where each vertex represents a state variable in 3D space. Subsequently, a weighted least squares algorithm is employed to iteratively estimate the state of the scanned workpiece based on the captured point cloud measurements. This framework offers the potential to incorporate information from diverse sensors into the CAD domain, facilitating a more comprehensive analysis. Preliminary results demonstrate promising performance, with the algorithm achieving convergence to a sub-millimeter standard deviation in the region of interest using only approximately 50 point cloud samples. This highlights the potential of utilising commercially available stereo cameras for high-precision quality control applications.
BibTeX
@inproceedings{vedove2025surface,
  author = {Matteo Dalle Vedove and Matteo Bonetto and Edoardo Lamon and Luigi Palopoli and Matteo Saveriano and Daniele Fontanelli},
  title = {{Surface Defect Identification using Bayesian Filtering on a 3D Mesh}},
  booktitle = {Measurement},
  year = {2025},
  doi = {10.1016/j.measen.2025.101849},
}
Figure