Image Segmentation Technique for Locating Automotive Parts on Belt Conveyors

TL;DR

This paper presents a robust, model-free image segmentation technique for reliably locating automotive parts on conveyor belts, even with noisy images and challenging backgrounds. The method combines standard image processing steps in a unique sequence to create clean silhouettes of objects, enabling accurate position and orientation determination for automated assembly.

What Problem Does This Paper Address?

Automated manufacturing processes, particularly in automotive assembly, rely heavily on robots accurately identifying and picking up parts. A critical challenge in this is reliably segmenting (isolating) these parts from their background, especially when they are presented on a moving conveyor belt. The authors highlight that simple thresholding of gray-level images often fails in industrial settings due to factors like noisy images, dirty conveyor belts, and variations in lighting. This can lead to fragmented object representations or background elements being incorrectly identified as parts, hindering precise robotic manipulation. The paper aims to provide a general-purpose solution that doesn't require specific models of the parts being recognized.

Methods

The core of the technique is a sequence of standard image processing operations applied in a specific order to achieve robust segmentation:

  • Edge Detection: A gradient operator (like Sobel) is used to enhance pixels along object boundaries, highlighting edges. The system allows for the selection of 12 different standard edge operators.
  • Smoothing: A 3x3 neighborhood averaging filter is applied to remove isolated "noise" edges in the background and to reinforce existing object edges.
  • Automatic Threshold Selection: The smoothed edge data is converted into a binary image. The threshold is automatically determined by finding the point of maximum second derivative on the histogram of the edge data, which is experimentally found to produce consistent results.
  • Gap Filling: To ensure object boundaries are closed, a process connects pixels that are within a small Euclidean distance (three pixels) of each other. This is achieved by checking complementary neighborhoods around each pixel.
  • Hole Filling through Connectivity Analysis: A 4-connectivity analysis is performed to fill in any "holes" (regions of background pixels) within the identified object silhouettes. Regions not connected to the image boundary are considered internal holes and are filled. Optionally, small regions or those touching the image boundary can be removed as noise.

Datasets

The paper describes testing the system on "several hundred scenes" of automotive parts on conveyor belts. It mentions using images of various resolutions (50x50, 100x100, 128x128, and 244x256) captured by different cameras. No specific named dataset is provided, nor is a new benchmark introduced.

Key Insights and Ideas

  • Sequential Application is Key: The effectiveness of the technique lies not just in the individual image processing steps but in their specific, sequential application. This order is crucial for overcoming the limitations of simple thresholding.
  • Model-Free Robustness: The approach is designed to be model-free, meaning it doesn't need prior knowledge of the exact shape or appearance of the parts. This makes it adaptable to a wide variety of objects and situations.
  • Handling Industrial Noise: The combination of edge enhancement, smoothing, and gap/hole filling effectively deals with common industrial image imperfections like noise, dirt, and fragmented edges.
  • Silhouettes for Location: The ultimate goal is to produce clean silhouettes, which contain sufficient information for determining the position and orientation of parts, even if internal features are not visible or reliable.

Key Results

  • The technique successfully segmented objects from the background, producing clean silhouettes.
  • The system was validated on a "large variety of parts and belt surfaces."
  • It was demonstrated to be capable of meeting "production rates" and has "potential for actual production use."
  • The speed of the program depends on implementation and image resolution, but meeting "several seconds" cycle time requirements is considered "no problem."
  • The accuracy of segmentation depended on image resolution, objects, backgrounds, and lighting conditions. Overlapping parts or parts "almost touching" could lead to meaningless silhouettes.

Limitations and Future Work

  • Overlapping Parts: The paper explicitly states that if parts overlap or are "almost touching," the program can produce meaningless silhouettes. This is a significant limitation for scenarios where parts might be presented in contact.
  • Image Resolution and Noise: While the technique is robust, its performance is still influenced by image resolution, object complexity, background clutter, and lighting conditions.
  • Future Work (Implied): The authors mention an "optional capability of ignoring such segmentations based on matching them to very simple models of parts expected." This suggests that incorporating simple model matching could further improve robustness and handle cases where segmentation alone might be ambiguous or incorrect. The integration into a prototype belt conveyor-robot arm system also points towards future work in real-world deployment and integration.

Summary updated

BibTeX
@inproceedings{1977image,
  title = {{Image Segmentation Technique for Locating Automotive Parts on Belt Conveyors}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {1977},
}
Figure