Contour Primitive of Interest Extraction Network Based on One-shot Learning for Object-Agnostic Vision Measurement

Fangbo Qin, Jie Qin, Siyu Huang, De Xu

Abstract

Image contour based vision measurement is widely applied in robot manipulation and industrial automation. It is appealing to realize object-agnostic vision system, which can be conveniently reused for various types of objects. We propose the contour primitive of interest extraction network (CPieNet) based on the one-shot learning framework. First, CPieNet is featured by that its contour primitive of interest (CPI) output, a designated regular contour part lying on a specified object, provides the essential geometric information for vision measurement. Second, CPieNet has the one-shot learning ability, utilizing a support sample to assist the perception of the novel object. To realize lower-cost training, we generate support-query sample pairs from unpaired online public images, which cover a wide range of object categories. To obtain single-pixel wide contour for precise measurement, the Gabor-filters based non-maximum suppression is designed to thin the raw contour. For the novel CPI extraction task, we built the Object Contour Primitives dataset using online public images, and the Robotic Object Contour Measurement dataset using a camera mounted on a robot. The effectiveness of the proposed methods is validated by a series of experiments.

The Challenge of Flexible Vision Measurement

In industrial automation and robotics, vision measurement is critical for determining the exact 3D pose or structure of an object. Unlike general computer vision tasks that might simply classify an object, measurement requires extreme precision. Engineers typically rely on contours—the crisp edges of a part—because they are mathematically predictable and robust to lighting changes.

However, traditional systems are often "object-specific." Measuring a new type of gear or bracket usually requires manual reprogramming or training a new model from scratch. This research addresses the challenge of object-agnostic measurement by creating a system that can identify specific "Contour Primitives of Interest" (CPI) on a brand-new object using just a single annotated example, known as a "support" image.

Figure 1: The one-shot learning framework for CPI extraction. (a) shows the extraction of a single contour primitive (CPI1), while (b) demonstrates the extraction of multiple primitives (CPI1-4) from a support image to a query image.

Methodology: The CPieNet Architecture

CPieNet operates on a one-shot learning framework using a dual-branch structure, similar to a Siamese network. One branch processes the support image (the annotated example), while the other processes the query image ( the new scene).

Prototype Extraction and Relevance Weighting

The network utilizes "Masked Average Pooling" to condense the annotated contour from the support image into a "prototype vector." This vector acts as a mathematical fingerprint of the specific edge. A key innovation is the Relevance Weighting (RW) module, which compares the prototype to the query image and weights the pixels accordingly. This allows the network to ignore background noise and irrelevant edges, focusing only on features that match the prototype.

Precision Thinning with GF-NMS

Standard neural networks often produce "fuzzy" edge predictions. For precise measurement, a line exactly one pixel wide is required. The researchers developed GF-NMS (Gabor-Filter based Non-Maximum Suppression), a thinning algorithm that detects edge direction to sharpen the raw output into a precise, single-pixel contour.

Synthetic Training Strategy

To overcome the need for massive labeled datasets, the authors developed a method to automatically generate training pairs from single public images. By applying random transformations and simulating lighting changes, they created a robust training pipeline that allows the model to generalize across a wide range of object categories.

Experimental Validation and Datasets

The performance of CPieNet was evaluated using two new benchmarks:

  • Object Contour Primitives (OCP): 2,307 web images of mechanical parts and electronics annotated with line segments and arcs.
  • Robotic Object Contour Measurement (ROCM): 523 real-world images captured by a robotic arm under varying viewpoints and lighting conditions.

Key Results

  • High Accuracy: CPieNet achieved a Maximum F-Measure (MF-ODS) of approximately 0.826 on the OCP dataset and 0.856 on robotic data, outperforming standard one-shot segmentation models like CANet and PANet.
  • Real-Time Performance: The system processes images in approximately 17.4 milliseconds, meeting the requirements for real-time robotic control loops.
  • Object-Agnostic Flexibility: By treating contour extraction as a matching problem, the system can identify features on novel objects without retraining, provided a single support example is available.

Conclusion and Future Directions

CPieNet represents a significant step toward flexible, object-agnostic robotic vision. While the system is highly effective, the authors noted that extreme perspective shifts (e.g., viewing a circle from a side angle where it appears as a line) remain a challenge. Future work will focus on improving robustness to 3D perspective changes and integrating the network into fully closed-loop robotic manipulation systems.

Summary updated

Abstract

Image contour based vision measurement is widely applied in robot manipulation and industrial automation. It is appealing to realize object-agnostic vision system, which can be conveniently reused for various types of objects. We propose the contour primitive of interest extraction network (CPieNet) based on the one-shot learning framework. First, CPieNet is featured by that its contour primitive of interest (CPI) output, a designated regular contour part lying on a specified object, provides the essential geometric information for vision measurement. Second, CPieNet has the one-shot learning ability, utilizing a support sample to assist the perception of the novel object. To realize lower-cost training, we generate support-query sample pairs from unpaired online public images, which cover a wide range of object categories. To obtain single-pixel wide contour for precise measurement, the Gabor-filters based non-maximum suppression is designed to thin the raw contour. For the novel CPI extraction task, we built the Object Contour Primitives dataset using online public images, and the Robotic Object Contour Measurement dataset using a camera mounted on a robot. The effectiveness of the proposed methods is validated by a series of experiments.
BibTeX
@inproceedings{qin2021contour,
  author = {Fangbo Qin and Jie Qin and Siyu Huang and De Xu},
  title = {{Contour Primitive of Interest Extraction Network Based on One-shot Learning for Object-Agnostic Vision Measurement}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2021},
  url = {https://openreview.net/forum?id=p__Zlh0PbkI},
}

Loading related content...

Figure