Exploring Figure-Ground Assignment Mechanism in Perceptual Organization

Wei Zhai, Yang Cao, Jing Zhang, Zheng-Jun ZhaNeurIPS 2022 · OpenReview

TL;DR

This paper introduces a Figure-Ground-Aided (FGA) module that empowers Convolutional Neural Networks (CNNs) with human-like perceptual organization. By explicitly teaching models to recognize "configural cues" like convexity and lower-region positioning, the authors significantly improve performance and data efficiency in challenging tasks where objects blend into their backgrounds, such as camouflaged object detection and medical imaging.

What Problem Does This Paper Address?

While modern CNNs excel at recognizing objects with clear visual boundaries, they often stumble when the foreground and background are visually similar—a problem known as visual ambiguity. In scenarios like camouflaged object detection (e.g., an owl blending into tree bark) or medical image segmentation (e.g., a polyp in a colon), standard discriminative learning often fails to "see" the object as a coherent unit.

When visual appearance differences between foreground and background are obscure (B), standard grouping fails. The proposed Figure-Ground Assignment mechanism (C) aids in resolving this ambiguity.

The authors argue that current AI lacks the figure-ground assignment mechanism inherent in human vision. Humans use geometric and contextual priors (Gestalt principles) to decide which side of a boundary belongs to the "figure" (the object) and which belongs to the "ground" (the background). This paper seeks to bridge this gap by incorporating these psychological priors directly into the neural network's architecture.

Methods

The core contribution is the Figure-Ground-Aided (FGA) module, which can be integrated into standard encoder-decoder architectures.

  1. Configural Cue Learning: Instead of relying solely on final segmentation masks, the FGA module is trained using two intermediate "configural cues" derived from ground-truth labels:
  2. Convexity: Identifying regions that "bulge" out (typically the object) versus those that "indent" (typically the background).
  3. Lower Region: Leveraging the human tendency to perceive the lower part of a divided region as the "figure."
Configural cues extracted from ground truth (GT): Lower Region (LR) and Convexity help the model understand geometric priors for figure-ground segregation.
  1. Interactive Enhancement Module (IEM): This module fuses these learned cues with standard image features. It uses a Collaborative Local Interaction (CLI) to align features at the boundaries and a Global Interaction (GI) to ensure object consistency.
  2. Lambda Strategy: To keep the self-attention mechanism computationally efficient, the authors utilize a "Lambda" layer, which captures long-range interactions without the quadratic cost of standard transformers.

Datasets and Evaluation

The authors evaluated their method across several challenging domains:

  • Figure-Ground Segregation (FGS) Test: A new synthetic benchmark designed by the authors. It uses random textures for both foreground and background to force the model to rely on shape and boundary logic rather than color or texture cues.
  • Camouflaged Object Detection (COD): Evaluated on CHAMELEON, CAMO, and the large-scale COD10K dataset.
  • Medical Image Segmentation: Tested on five polyp segmentation datasets (Kvasir, CVC-612, etc.) and a COVID-19 lung infection CT dataset.
  • Salient Object Detection: Assessed using DUTS-Test and PASCAL-S.

Key Insights and Results

  • Inductive Bias via Psychophysics: Teaching a model how to look (via convexity and lower-region cues) is more effective than just telling it what the final answer is. These cues provide a strong inductive bias that helps resolve visual ambiguity.
  • Extreme Data Efficiency: The FGA-augmented model is remarkably fast at learning. Experiments showed that a model trained on just 1% of the data with FGA could match the performance of a baseline model trained on 10% of the data with standard edge supervision.
  • State-of-the-Art Performance: The FGA-augmented models consistently outperformed previous methods. On the COD10K dataset, the FGA model achieved an S-measure of 0.821, surpassing the previous SOTA (SINetv2 at 0.815). In COVID-19 lung infection segmentation, the model reached a Dice score of 0.754, a significant jump over the next best method (0.700).

Limitations and Future Work

This work focuses primarily on convexity and lower-region cues. Other powerful Gestalt principles, such as symmetry, parallelism, and "meaningfulness," remain unexplored in this framework. Future directions could involve integrating more complex top-down knowledge (like object-specific shape priors) to further enhance the model's ability to resolve extreme ambiguity. The authors also note the societal impact of high-performance segmentation, which could be misused in unauthorized surveillance or poaching scenarios where camouflage is bypassed.

Summary updated

Abstract

Perceptual organization is a challenging visual task that aims to perceive and group the individual visual element so that it is easy to understand the meaning of the scene as a whole. Most recent methods building upon advanced Convolutional Neural Network (CNN) come from learning discriminative representation and modeling context hierarchically. However, when the visual appearance difference between foreground and background is obscure, the performance of existing methods degrades significantly due to the visual ambiguity in the discrimination process. In this paper, we argue that the figure-ground assignment mechanism, which conforms to human vision cognitive theory, can be explored to empower CNN to achieve a robust perceptual organization despite visual ambiguity. Specifically, we present a novel Figure-Ground-Aided (FGA) module to learn the configural statistics of the visual scene and leverage it for the reduction of visual ambiguity. Particularly, we demonstrate the benefit of using stronger supervisory signals by teaching (FGA) module to perceive configural cues, \ie, convexity and lower region, that human deem important for the perceptual organization. Furthermore, an Interactive Enhancement Module (IEM) is devised to leverage such configural priors to assist representation learning, thereby achieving robust perception organization with complex visual ambiguities. In addition, a well-founded visual segregation test is designed to validate the capability of the proposed FGA mechanism explicitly. Comprehensive evaluation results demonstrate our proposed FGA mechanism can effectively enhance the capability of perception organization on various baseline models. Nevertheless, the model augmented via our proposed FGA mechanism also outperforms state-of-the-art approaches on four challenging real-world applications.
BibTeX
@inproceedings{zhai2022exploring,
  author = {Wei Zhai and Yang Cao and Jing Zhang and Zheng-Jun Zha},
  title = {{Exploring Figure-Ground Assignment Mechanism in Perceptual Organization}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2022},
  url = {https://openreview.net/forum?id=c3HrNgQE7d},
}

Loading related content...

Figure