Grouping Contours by Iterated Pairing Networks

Amnon Shashua, Shimon UllmanNeurIPS 1990 · OpenReview

TL;DR

This paper introduces a novel neural network architecture for grouping image contour fragments into meaningful shapes. It proposes an iterative process of "saliency" and "pairing" stages that allows local decisions to propagate and form globally optimal groupings based on length and smoothness, mimicking perceptual organization.

What Problem Does This Paper Address?

Visual systems are remarkably adept at grouping fragmented lines and curves into coherent shapes, a process crucial for object recognition and scene understanding. This "contour grouping" is often achieved spontaneously, without explicit scanning or knowledge of object identity. Traditional approaches struggled to capture this emergent, global optimization from local cues. The paper addresses the challenge of creating a computational model that can autonomously group contour fragments into perceptually salient structures, prioritizing longer, smoother curves, and doing so in a way that can be implemented on a simple, locally connected network.

Methods

The core of the proposed method is an iterative network that operates in two stages:

  • Saliency Stage: Each contour element (pixel) computes a "saliency" value. This is an upper-bound on how salient a curve passing through that element can be. It's computed iteratively, where each element considers the saliency contributions from its neighbors, effectively "filling in" gaps and favoring smooth paths.
  • Pairing Stage: At each node (representing a local region), elements are paired up. This pairing is based on "mutual neighbors" – elements that maximally contribute to each other's saliency. This local pairing decision is then used to update the saliency values in the next iteration.

These two stages are iterated, allowing pairing decisions to propagate and refine saliency values, leading to the emergence of globally salient groups. The entire process is framed as an optimization problem aiming to maximize a quality measure for each group, defined by its length and smoothness.

Datasets

Not discussed in the paper. The paper uses illustrative examples of contour images, including those from Figure 1 (a car, house, trees, and texture) and Figure 2 (various abstract shapes and contour arrangements).

Key Insights and Ideas

  • Saliency as an Upper-Bound: The concept of using a local saliency measure as an upper-bound for global grouping quality is a clever way to guide the optimization process.
  • Iterated Pairing for Global Optimization: The iterative nature of pairing and saliency updates allows local decisions to influence and refine global structure, mimicking how perceptual grouping can resolve ambiguities.
  • Biologically Plausible Network: The proposed computation maps onto a uniform, locally connected network, making it potentially implementable in biological neural systems.
  • Gap Filling as a By-product: The saliency computation inherently fills in gaps in contours, a critical aspect of perceptual grouping that is achieved naturally by the iterative process.
  • Prioritizing Smoothness and Length: The quality measure explicitly favors longer and smoother contours, aligning with Gestalt principles of perceptual organization.

Key Results

The paper demonstrates the effectiveness of the iterated pairing network through qualitative examples. Figure 2 shows that after 30 iterations, the network successfully identifies and highlights perceptually salient groups in various contour images. The saliency map visually corresponds to the brightness and width of the detected groups, indicating that stronger groups are associated with higher saliency. The paper suggests that these identified groups closely correspond to objects of interest in the images.

Limitations and Future Work

  • Convergence to Global Optimum: The paper acknowledges that the process is not guaranteed to converge to a globally optimal solution, though it yields good results in practice.
  • Computational Complexity: While the network is described as simple and locally connected, the paper mentions that finding an optimal weighted match (related to the pairing stage) has a polynomial algorithm (Edmonds' algorithm), suggesting that efficient implementation is possible but might still be computationally intensive for very large images.
  • Parameter Sensitivity: The paper mentions the role of parameters like 'n' (number of preceding elements considered for saliency) and 'a' (in the coupling constant), but the optimal selection and sensitivity of these parameters are not deeply explored.
  • Future Work: The authors suggest that the described computation is a "biologically plausible scheme" and that further exploration of its implementation and behavior in more complex scenarios would be valuable. The paper also hints at the potential for this method to be extended to more general segmentation and object recognition tasks.

Summary updated

Abstract

Shimon Ullman We describe in this paper a network that performs grouping of image con(cid:173) tours. The input to the net are fragments of image contours, and the output is the partitioning of the fragments into groups, together with a saliency measure for each group. The grouping is based on a measure of overall length and curvature. The network decomposes the overall opti(cid:173) mization problem into independent optimal pairing problems performed at each node. The resulting computation maps into a uniform locally connected network of simple computing elements. 1 The Problenl: Contour Grouping A problem that often arises in visual information processing is the linking of con(cid:173) tour fragments into optimal groups. For example, certain subsets of contours spon(cid:173) taneously form perceptual groups, as illustrated in Fig. 1, and are often detected immediately without scanning the image in a systematic manner. Grouping process of this type are likely to play an important role in object recognition by segmenting the image and selecting image structures that are likely to correspond to objects of interest in the scene. 'Ve propose that some form of autonomous grouping is performed at an early stage based on geometrical characteristics, that are independent of the identity of objects to be selected. The grouping process is governed by the notion of saliency in a way that priority is given to forming salient groups at the expense of potentially less salient ones. This general notion can again be illustrated by Fig. 1; it appears that certain groups spontaneously emerge, while grouping decisions concerning the less salient parts of the image may remain unresolved. As we shall see, the computation below exhibits a similar behavior. We define a grouping of the image contours as the formation of a set of disjoint
BibTeX
@inproceedings{shashua1990grouping,
  author = {Amnon Shashua and Shimon Ullman},
  title = {{Grouping Contours by Iterated Pairing Networks}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {1990},
  url = {https://openreview.net/forum?id=S1bRJwZdZH},
}

Loading related content...

Figure