TL;DR
This paper presents an FPGA-based algorithm for detecting conveyor belt deviations in underground mining robots. By optimizing the traditional Line Segment Detector (LSD) algorithm for hardware implementation, the authors achieve real-time, high-precision detection crucial for preventing costly accidents and operational downtime.
What Problem Does This Paper Address?
Conveyor belts are critical for bulk material transport in industries like mining. However, belt deviation is a common and dangerous fault that can lead to material spillage, increased wear, and even fire or belt breakage. Traditional methods for detecting this deviation often rely on dense sensor arrangements or software-based image processing. While software approaches offer flexibility, they struggle with the real-time processing demands of video streams, leading to high false detection rates and limiting their applicability on mobile platforms like inspection robots. This paper addresses the need for a robust, real-time, and hardware-efficient solution for conveyor belt deviation detection, specifically for an underground inspection robot.
Methods
The core contribution of this paper is the development of an FPGA-based Line Segment Detector (LSD) IP core. The authors adapt the traditional LSD algorithm, which involves image filtering, gradient calculation, region growing, and rectangle approximation, for efficient hardware implementation. Key methodological aspects include:
- FPGA Implementation: Leveraging the parallel processing capabilities of FPGAs (specifically Xilinx XC7Z020-2CLG484I SoC) to accelerate the computationally intensive parts of the LSD algorithm.
- Hardware-Optimized Modules:
- Image Preprocessing: Implementing histogram equalization and Gaussian filtering using hardware modules to improve real-time performance and reduce logic resources.
- Edge Detection: Developing an edge detection IP core that includes gradient information calculation, non-maximum value suppression, and adaptive thresholding (Otsu's method) for robust edge linking.
- Region Growth: Designing a novel region growth module that uses linearly increasing column numbers as virtual seeds and incorporates a mirroring module to handle line segments in different orientations.
- Real-time Performance: The design focuses on minimizing latency through pipeline processing and fixed-point arithmetic, crucial for the robot's operational speed.
Datasets
The paper uses two types of data for evaluation:
- Synthetic Data: A noise-free, hand-drawn image with line segments divided at π/8 intervals was used to initially verify the algorithm's ability to detect line segments at various angles and suppress edges outside a specific angular range.
- Real-world Data: Top-plan view images of a belt conveyor operation collected by an IMX222 camera were used to test the algorithm's stability and performance in a noisy, real-world environment.
Key Insights and Ideas
- FPGA for Real-time Vision: The paper demonstrates that FPGAs are well-suited for accelerating complex image processing algorithms like line segment detection, enabling real-time performance that is difficult to achieve with software alone on embedded systems.
- Optimized LSD for Hardware: Adapting the LSD algorithm by breaking it down into hardware-specific modules (preprocessing, edge detection, region growth) allows for significant speedups and resource optimization.
- Mirroring for Orientation Robustness: The introduction of a mirroring module before region growth is a clever way to ensure that line segments oriented in opposite directions to the primary scanning direction are still detected.
- Adaptive Thresholding for Edge Linking: Using Otsu's method for adaptive thresholding in edge linking improves robustness to varying lighting conditions and image noise.
- Virtual Seeds in Region Growth: The proposed method of using linearly increasing column numbers as virtual seeds simplifies the seed selection process for region growth on an FPGA.
Key Results
- High Frame Rate: The FPGA-based algorithm achieves a processing frame rate of 108 FPS at a 100 MHz clock frequency, significantly outperforming the original LSD algorithm running on a CPU (AMD Ryzen 9 3950X), which achieved an average processing time of 9.2168 ms per frame compared to the FPGA's 20.226 ms.
- Accurate Line Segment Detection: The algorithm accurately detects line segments with edge direction angles from 45° to 135° in both synthetic and real-world images.
- Good Stability: The algorithm exhibits good stability in real images with noise.
- Moderate Resource Occupancy: The FPGA implementation shows reasonable resource utilization, with Slice LUTs at 6.17%, FF at 3.70%, BRAM at 21.07%, and DSP at 5.45% of available resources.
Limitations and Future Work
- Fixed-Point Arithmetic Errors: The paper acknowledges that the fixed-point number calculation method and pipelined processing architecture can lead to certain errors in the detected line segment information compared to the original LSD algorithm.
- Limited Angle Range: While the algorithm is effective for angles between 45° and 135°, its performance for line segments outside this range might be less optimal without further modifications.
- Future Work: The authors suggest that further research could focus on improving the accuracy of the detected line segment information, potentially by exploring more advanced fixed-point representations or alternative algorithm structures. Expanding the detection capabilities to other types of conveyor belt faults could also be a future direction.
Summary updated