Accurate Detection of Weld Seams for Laser Welding in Real-World Manufacturing

Rabia Ali, Muhammad Sarmad, Jawad Tayyub, Alexander VogelAAAI 2023 · DOI

TL;DR

This paper introduces a novel deep learning system for accurately detecting weld seams in real-world manufacturing settings, overcoming the limitations of traditional computer vision methods. By employing a two-module approach (edge classification and edge detection) and a modified U-Net architecture, the system achieves pixel-level precision, enabling fully automated laser welding without human intervention.

What Problem Does This Paper Address?

Laser welding is a crucial, high-precision manufacturing process, but its accuracy hinges on the precise detection of the weld seam. Traditional methods like Canny or Hough edge detectors, while useful, lack the stringent tolerance required for industrial applications, leading to frequent manual verification and correction by human operators. This manual intervention is time-consuming, labor-intensive, and prone to human error, especially after long working hours. The challenge is to develop an automated system that can reliably detect weld seams with sub-pixel accuracy in complex, real-world industrial environments, even for narrow seams where traditional methods struggle.

Methods

The proposed solution is a two-module deep learning pipeline:

  • Edge Classification Module (C): This module acts as a pre-classifier, using a simple binary classifier (likely a CNN like AlexNet or ResNet) to distinguish between "normal" (clear, detectable weld seam) and "abnormal" (unclear, blurry, or already welded) images. This filters out problematic images, preventing the subsequent module from processing them and thus improving overall system robustness.
  • Edge Detection Module (D): This module utilizes a modified U-Net architecture. The key modification involves replacing the 3x3 convolution in the penultimate layer with a 2x2 max-pooling layer to produce a 1D output. This simplification is justified by the fact that weld seams in their domain are essentially vertical lines. The output is a binary edge mask, where the transition from 0s to 1s indicates the weld seam.

The system is trained end-to-end, with the classification module first filtering images, and then the edge detection module processing only the "normal" images.

Datasets

The authors introduce a new dataset called the Laser Seam Detection (LSD) dataset.

  • It consists of 37,952 real-world welding images of size 656x494.
  • The dataset includes images categorized as 'clear', 'unclear' (visually difficult to locate), and 'welded' (already processed).
  • A subset of 700 'clear' and 700 'unclear' images were used for training the classification module to ensure a balanced dataset.
  • For the edge detection module, images with 'clear' and 'unclear' seams were used for training, while 'welded' images were excluded from training but used in the test set for evaluation.
  • The dataset was manually labeled using the Computer Vision Annotation Tool (CVAT) and verified by professional welding engineers.
  • The authors plan to make the dataset and code publicly available.

Key Insights and Ideas

  • The importance of pre-classification: Filtering out "abnormal" images (unclear or already welded seams) significantly improves the performance and robustness of the edge detection module. This is crucial for real-world industrial scenarios where image quality can vary.
  • Simplified U-Net for vertical edges: Modifying the U-Net architecture to output a 1D mask by replacing a convolution with max-pooling is an effective simplification for detecting vertical weld seams, reducing complexity without sacrificing accuracy.
  • Data saturation: The accuracy of the edge detection module plateaus after approximately 1500 training images, suggesting that extensive data collection beyond this point might yield diminishing returns for this specific task.
  • CPU-only inference is sufficient: The proposed system achieves good inference times on a CPU, eliminating the need for expensive GPU hardware for deployment, which is a significant advantage for industrial integration.

Key Results

  • Edge Classification: The classification module achieved high accuracies, with ResNet-18 performing best at 97.8% for clear/unclear classification and AlexNet at 96.0% for welded/non-welded classification.
  • Edge Detection: The modified U-Net architecture achieved state-of-the-art performance.
  • Without the edge classification module, the system achieved 91.7% accuracy at ±1 pixel tolerance.
  • With the edge classification module, accuracy improved to 98.0% at ±1 pixel tolerance, demonstrating the effectiveness of the filtering step.
  • At ±3 pixel tolerance, the system achieved 99.2% accuracy.
  • Inference Time: The system achieves an inference time of less than 200 ms per job on a CPU, which is comparable to or faster than human operators.
  • Dataset Size: The authors found that around 1500 training images were sufficient for the edge detection module to reach its performance plateau.

Limitations and Future Work

  • Limitations:
    • The paper notes that adding image filters like Gaussian or Adaptive Thresholding did not improve performance and sometimes degraded it, possibly due to visual artifacts on the metallic surface confusing the network.
  • The current system is trained on images from a specific laser welding machine (Trumpf TruLeserCell 3000/5000) and may require retraining or fine-tuning for different machine types or environments.
  • Future Work:
    • The authors suggest that the system's consistent performance over several months indicates its suitability for full transition to an automated U-Net system, relieving human operators of manual tasks.
  • The paper contributes a valuable real-world dataset and code, encouraging further research in industrial computer vision for welding applications.
  • The authors mention the potential for further exploration into different preprocessing techniques or network architectures.

Summary updated

BibTeX
@inproceedings{ali2023accurate,
  author = {Rabia Ali and Muhammad Sarmad and Jawad Tayyub and Alexander Vogel},
  title = {{Accurate Detection of Weld Seams for Laser Welding in Real-World Manufacturing}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2023},
  doi = {10.1609/aaai.v37i13.26834},
}
Figure