A Multi-Sensor Data Fusion System for Laser Welding Process Monitoring

Fuqin Deng, Yongshen Huang, Song Lu, Yingying Chen, Jia Chen, Hua Feng, Jianmin Zhang, Yong Yang, Junjie Hu, Tin Lun Lam, Fengbin XiaIEEE Access 2020 · OpenReview

Abstract

Most existing laser welding process monitoring (LWPM) technologies focus on detecting post-process defects. However, in sheet metal laser welding applications such as welding of electronic consumer products during mass production, in-process defect detection is more important. In this article, a compact LWPM system using multi-sensor data fusion to detect in-process defects has been built. This system can collect the time series of plasma intensity, light intensity and temperature data for feature analysis. To verify the system’s effectiveness, a plasma-light-temperature dataset has been compiled, which consists of 5,836 samples of nine classes, including one positive class and eight negative classes of typical in-process defects. A multi-sensor data fusion network based on a convolution neural network for in-process defect detection, called IDDNet, has also been proposed. Experimental results have demonstrated that IDDNet can achieve better multi-classification results than the support vector machine, with an overall accuracy of 97.57%. In particular, considering this monitoring process as a binary classification problem, IDDNet can achieve a 99.42% accuracy. Moreover, IDDNet can reach an average speed of 0.79ms per sample on a single GTX 1080ti graphics card, which meets the real-time requirement for industrial production. The proposed LWPM system has been successfully verified in real applications of sheet metal laser welding.

Introduction

Laser welding is a cornerstone of modern electronics and automotive manufacturing, prized for its high precision and speed. However, the process is inherently volatile. Minor deviations—such as a tilted workpiece, surface impurities like glue or water, or a laser defocusing—can lead to significant structural failures.

Traditionally, manufacturers have relied on post-process monitoring, which inspects parts after production is complete. This approach risks "batch failure," where a machine misalignment at the start of a shift results in hundreds of defective parts before detection. While visual camera-based systems offer real-time monitoring, they are often computationally expensive, data-intensive, and susceptible to interference from the intense laser light. This research introduces a compact, high-speed solution designed for robust in-process defect detection.

System Design and Methodology

Hardware and Sensor Fusion

The proposed system utilizes a hardware-software co-design integrated into a Han’s Laser welding machine. Rather than relying on bulky cameras, the system integrates three lightweight sensors to capture critical process data:

  • Photodiodes: Two sensors measure plasma intensity and reflected light intensity.
  • Pyrometer: One sensor monitors the thermal radiation (temperature) of the weld pool.
The proposed LWPM system: (a) 3D model of the hardware integration; (b) Schematic diagram of the optical path and signal collection process.

IDDNet Architecture

To process the multi-sensor data, the researchers developed the In-Process Defect Detection Network (IDDNet), a 1D Convolutional Neural Network (CNN). Unlike traditional machine learning methods like Support Vector Machines (SVM) that require manual feature engineering, IDDNet automatically extracts complex patterns from raw time-series data. The architecture consists of three convolution blocks—incorporating Batch Normalization and ReLU activation—followed by Global Average Pooling, optimized for 128-point time-series signals.

Experimental Setup and Dataset

The system was evaluated using the Plasma-Light-Temperature Dataset (PLTD), which contains 5,836 samples. The dataset covers nine distinct classes: one "Qualified" class for perfect welds and eight specific defect types, including Defocus, White Glue, Missing Weld, Drift, Tilt, Repetition, and Water contamination. Each sample comprises three parallel time-series signals, each 128 points in length.

Results and Discussion

Classification Performance

IDDNet demonstrated superior performance compared to traditional methods. While SVMs achieved approximately 83% accuracy, IDDNet reached a multi-class accuracy of 97.57%. When simplified to a binary classification problem (detecting any defect vs. a qualified weld), the system achieved 99.42% accuracy.

Analysis revealed that while temperature is the most predictive single variable (~90% accuracy), the fusion of all three sensors is essential for reaching the 97%+ threshold. Plasma and light intensity provide critical context that thermal data alone lacks.

Real-Time Industrial Viability

For industrial applications, processing speed is as critical as accuracy. IDDNet processes a single sample in just 0.79ms on a standard GTX 1080ti GPU. Given that the welding process itself occurs on a millisecond scale, this performance allows for near-instantaneous defect flagging, preventing the production of substandard parts in high-speed lines.

Conclusion

The proposed LWPM system successfully demonstrates that multi-sensor data fusion combined with deep learning can provide a robust, real-time solution for laser welding quality control. Future work will focus on improving model robustness across varying power levels and speeds, as well as developing closed-loop systems that can automatically adjust laser parameters to prevent defects as they begin to form.

Summary updated

Abstract

Most existing laser welding process monitoring (LWPM) technologies focus on detecting post-process defects. However, in sheet metal laser welding applications such as welding of electronic consumer products during mass production, in-process defect detection is more important. In this article, a compact LWPM system using multi-sensor data fusion to detect in-process defects has been built. This system can collect the time series of plasma intensity, light intensity and temperature data for feature analysis. To verify the system’s effectiveness, a plasma-light-temperature dataset has been compiled, which consists of 5,836 samples of nine classes, including one positive class and eight negative classes of typical in-process defects. A multi-sensor data fusion network based on a convolution neural network for in-process defect detection, called IDDNet, has also been proposed. Experimental results have demonstrated that IDDNet can achieve better multi-classification results than the support vector machine, with an overall accuracy of 97.57%. In particular, considering this monitoring process as a binary classification problem, IDDNet can achieve a 99.42% accuracy. Moreover, IDDNet can reach an average speed of 0.79ms per sample on a single GTX 1080ti graphics card, which meets the real-time requirement for industrial production. The proposed LWPM system has been successfully verified in real applications of sheet metal laser welding.
BibTeX
@inproceedings{deng2020multi,
  author = {Fuqin Deng and Yongshen Huang and Song Lu and Yingying Chen and Jia Chen and Hua Feng and Jianmin Zhang and Yong Yang and Junjie Hu and Tin Lun Lam and Fengbin Xia},
  title = {{A Multi-Sensor Data Fusion System for Laser Welding Process Monitoring}},
  booktitle = {IEEE Access},
  year = {2020},
  url = {https://openreview.net/forum?id=45njY72NLa},
}
Figure