TL;DR
This paper introduces a novel image-based method for detecting bolt loosening in flange joints. By employing a checkerboard pattern for perspective correction, the method significantly improves the accuracy and stability of bolt loosening detection, even under challenging camera angles and lighting conditions, outperforming existing techniques.
What Problem Does This Paper Address?
Ensuring the structural integrity of bolted joints is crucial in many engineering applications. Manual bolt inspection is time-consuming, inefficient, and prone to human error, especially in large-scale structures. While various automated methods exist, such as guided waves and piezoelectric sensors, they often involve complex hardware or high costs. Computer vision offers a promising alternative, but accurately detecting bolt loosening from images is hampered by perspective distortion and camera position deviations. Existing image correction methods, often relying on rectangular gaskets, struggle to effectively address these issues, particularly with unique flange joint geometries and varying camera viewpoints. This paper aims to bridge this gap by proposing a robust perspective correction method specifically tailored for flange joint bolt inspection.
Methods
The core of this paper lies in a novel checkerboard-based perspective correction method. The approach involves placing a checkerboard pattern near the bolt of interest. This checkerboard serves as a reference for estimating a homography matrix. Unlike previous methods that used a minimum of four feature points (e.g., corners of a gasket or bolt group), this method leverages multiple corner points of the checkerboard. This increased number of reference points enhances the stability and accuracy of the homography matrix estimation.
The process involves:
- Camera Calibration: Correcting for lens distortion in the original image using internal camera parameters.
- Checkerboard Corner Detection: Automatically identifying the corner coordinates of the checkerboard in the corrected image.
- Homography Matrix Calculation: Computing the homography matrix using the detected checkerboard corner points. The paper emphasizes that using more than four points leads to a more accurate homography matrix.
- Image Correction: Applying the calculated homography matrix to rectify perspective distortion and camera position deviations in the bolt image.
- Bolt Loosening Diagnosis: After correction, standard image processing techniques (Canny edge detection and Hough transform) are used to identify the nut edge, and the bolt rotation angle is calculated to diagnose loosening.
Datasets
The study primarily uses laboratory-generated datasets. Images of flange joints with bolts were captured using an iPhone 12 under various controlled conditions, including different bolt loosening angles, camera positions (angles), camera heights, and lighting conditions (sunny and cloudy). The paper does not introduce a new public benchmark dataset but rather uses its experimental setup to generate data for validation and comparison.
Key Insights and Ideas
- Checkerboards are superior to rectangular gaskets for perspective correction: The increased number of feature points provided by a checkerboard significantly improves the robustness of homography estimation, especially when dealing with complex geometries and varying camera angles.
- More correction points lead to better stability and accuracy: The study demonstrates that using more than four checkerboard corner points consistently improves the accuracy and stability of the bolt image correction and subsequent loosening detection.
- Effective correction across a wide range of camera angles: The proposed method can correct camera position angle deviations from 0° to 180°, a significant improvement over existing methods that struggle beyond 45°.
- Low sensitivity to light intensity: The method performs reliably under both sunny and cloudy conditions, indicating its practicality for real-world applications where lighting can be variable.
- Optimal camera shooting distance: A sweet spot for camera height exists between 30 cm and 40 cm for achieving the best balance of accuracy and stability.
Key Results
- The proposed checkerboard-based method effectively corrects image perspective distortion and camera position deviation.
- The average error in bolt loosening detection for the corrected images is consistently less than 1.5 degrees, even with as few as nine correction points.
- In contrast, the average error for uncorrected images can reach up to 5.9 degrees, and this error increases significantly with perspective angle.
- The method demonstrates superior performance compared to the rectangular-spacer-based correction method, especially at a 90° camera position deviation angle, where the rectangular method fails to correct effectively.
- The recommended camera shooting distance range for optimal results is 20 cm to 60 cm.
- The method shows low sensitivity to light intensity, with minimal differences in accuracy and stability between sunny and cloudy conditions.
Limitations and Future Work
- Minimum correction points: While the method works with as few as nine correction points, the paper suggests that at least nine are needed for effective application.
- Physical checkerboard durability: The paper notes that stickers are easily peeled off due to corrosion, and suggests using waterproof and corrosion-resistant paint for the checkerboard. It also mentions potential interference from ice and snow.
- Optimal camera height: While the method is robust, the best accuracy and stability are achieved within a specific camera height range (30-40 cm). Deviations outside this range can lead to increased errors or decreased stability.
- Future work: The paper implicitly suggests further exploration into optimizing checkerboard design and placement for even greater robustness and ease of application in diverse industrial environments. The recommendation for waterproof and corrosion-resistant checkerboards also points towards practical implementation considerations.
Summary updated