Contour Primitive of Interest Extraction Network Based on One-Shot Learning for Object-Agnostic Vision Measurement

Fangbo Qin, Jie Qin, Siyu Huang, De Xu

TL;DR

Researchers have developed CPieNet, a deep learning network that can identify and extract specific geometric contours (like lines or arcs) from objects it has never seen before. By using a "one-shot" learning approach—where the system is shown just a single annotated example of a part—it can precisely locate that same feature in new images, enabling flexible and "object-agnostic" robotic vision measurement.

What Problem Does This Paper Address?

In industrial automation and robotics, "vision measurement" is the process of using cameras to determine the exact 3D pose or structure of an object. Unlike general computer vision (which might just label an image as "a bolt"), measurement requires extreme precision. Engineers usually rely on contours—the crisp edges of a part—because they are mathematically predictable and robust to lighting changes.

The bottleneck? Traditional systems are "object-specific." If you want to measure a new type of gear or bracket, you often have to manually program the feature extraction or train a new model from scratch. This paper tackles the challenge of object-agnostic measurement: creating a system that can walk up to a brand-new object and, with just one example (a "support" image), extract the specific "Contour Primitive of Interest" (CPI) needed for the task.

Methods

The authors introduce CPieNet, which operates on a one-shot learning framework. Here’s how the "magic" happens:

  1. The Architecture: The network uses a dual-branch structure (similar to a Siamese network). One branch processes a support image (with a known, annotated contour), and the other processes the query image (the new scene).
  2. Prototype Extraction: Using "Masked Average Pooling," the network condenses the annotated contour from the support image into a "prototype vector"—essentially a mathematical fingerprint of what that specific edge looks like.
  3. Relevance Weighting (RW): This is a key innovation. The network compares the prototype to the query image and "weights" the pixels. It effectively tells the query branch: "Ignore the background and other edges; only pay attention to things that look like our prototype."
  4. GF-NMS (Gabor-Filter based Non-Maximum Suppression): Standard neural networks produce "fuzzy" edges. For measurement, you need a line exactly one pixel wide. The authors designed a thinning algorithm using Gabor filters that detects the direction of the edge to sharpen it into a precise, single-pixel contour.
  5. Synthetic Training: To avoid the "data hunger" of deep learning, they developed a method to automatically generate training pairs from single public images by applying random transformations, mimics of "nearby stuff," and lighting changes.

Datasets

The authors introduced two new benchmarks to the field:

  • OCP (Object Contour Primitives): A collection of 2,307 images from the web featuring mechanical parts, household items, and electronics, annotated with line segments and circular arcs.
  • ROCM (Robotic Object Contour Measurement): A real-world dataset of 523 images across 15 objects, captured by a camera mounted on a robotic arm (eye-in-hand) under varying viewpoints and lighting.

Key Insights and Ideas

  • Task-Awareness over General Edges: Most edge detectors find every edge. CPieNet is "task-aware"—it only finds the specific edge you told it you cared about in the support image.
  • The Power of Relevance: The "Relevance Weighting" module allows the model to distinguish between very similar features. For example, it can tell the difference between a smooth edge and a threaded edge on the same bolt because the prototype captures those subtle textural differences.
  • One-Shot Flexibility: By treating contour extraction as a matching problem rather than a classification problem, the system becomes "object-agnostic." It doesn't need to know what a "fuel injector" is; it just needs to find the circle that looks like the one in the example.

Key Results

  • High Accuracy: CPieNet achieved an MF-ODS (Maximum F-Measure) of approximately 0.826 on the OCP dataset and up to 0.856 on robotic data, outperforming existing one-shot segmentation models like CANet and PANet.
  • Real-Time Speed: The system processes images in about 17.4 milliseconds, making it fast enough for real-time robotic control loops.
  • Thinning Efficiency: The GF-NMS thinned the raw output into 1-pixel lines with higher geometric fidelity than standard gradient-based methods.

Limitations and Future Work

While CPieNet is powerful, the authors note a few hurdles:

  • Extreme Viewpoints: The system struggles if the query image is taken from a vastly different angle than the support image (e.g., looking at a circle from the side so it appears as a thin line).
  • Cluttered Backgrounds: Very complex environments can still confuse the relevance weighting, leading to "false positive" edges.
  • Future Directions: The team plans to improve the model's robustness to 3D perspective shifts and integrate it into a fully closed-loop robotic measurement and manipulation system.

Summary updated

Abstract

Image contour based vision measurement is widely applied in robot manipulation and industrial automation. It is appealing to realize object-agnostic vision system, which can be conveniently reused for various types of objects. We propose the contour primitive of interest extraction network (CPieNet) based on the one-shot learning framework. First, CPieNet is featured by that its contour primitive of interest (CPI) output, a designated regular contour part lying on a specified object, provides the essential geometric information for vision measurement. Second, CPieNet has the one-shot learning ability, utilizing a support sample to assist the perception of the novel object. To realize lower-cost training, we generate support-query sample pairs from unpaired online public images, which cover a wide range of object categories. To obtain single-pixel wide contour for precise measurement, the Gabor-filters based non-maximum suppression is designed to thin the raw contour. For the novel CPI extraction task, we built the Object Contour Primitives dataset using online public images, and the Robotic Object Contour Measurement dataset using a camera mounted on a robot. The effectiveness of the proposed methods is validated by a series of experiments.
BibTeX
@inproceedings{qin2021contour,
  author = {Fangbo Qin and Jie Qin and Siyu Huang and De Xu},
  title = {{Contour Primitive of Interest Extraction Network Based on One-Shot Learning for Object-Agnostic Vision Measurement}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2021},
  url = {https://openreview.net/forum?id=p__Zlh0PbkI},
}

Loading related content...

Figure