A Salient Object Detection Method Based on Boundary Enhancement

Falin Wen, Qinghui Wang, Ruirui Zou, Ying Wang, Fenglin Liu, Yang Chen, Linghao Yu, Shaoyi Du, Chengzhi Yuan2023 · DOI

TL;DR

This paper introduces a novel method for salient object detection that significantly improves accuracy by focusing on enhancing object boundaries and effectively handling variations in object scale. By integrating multi-level feature aggregation, multi-scale information extraction, and a dedicated boundary extraction module, the approach achieves state-of-the-art performance on several benchmark datasets.

What Problem Does This Paper Address?

Visual saliency, the ability to quickly focus on important parts of a visual field, is crucial for tasks like medical imaging and robotics. Salient Object Detection (SOD) aims to automatically identify and localize these conspicuous regions in images. However, existing methods struggle with several challenges:

  • Scale Variation: Objects can appear at vastly different sizes, making it difficult for models to capture them consistently.
  • Boundary Ambiguity: Saliency maps often have blurred or imprecise boundaries, hindering accurate segmentation.
  • Information Loss: Deep convolutional neural networks (CNNs) can lose fine-grained details during downsampling, impacting the detection of small or intricate objects.
  • Data Imbalance: The sparsity of boundary pixels in training data can lead to biased learning.

This paper tackles these issues by proposing a method that explicitly models and enhances object boundaries while better integrating multi-scale and multi-level feature information.

Methods

The proposed method builds upon a Feature Pyramid Network (FPN) architecture and introduces several key innovations:

  • Multi-Level Feature Aggregation Module: This module enhances the expressive power of features at different resolutions by allowing adjacent features to complement each other. It addresses scale variation by aggregating information across different levels.
  • Multi-Scale Information Extraction Module: Integrated into the decoder, this module captures local contextual information at various scales. This helps to better understand feature map composition after fusion and improves the ability to handle targets of different sizes. It uses dilated convolutions with different dilation rates to extract features with varying receptive fields.
  • Boundary Extraction Module: This novel module explicitly models the boundaries of salient objects. It extracts boundary information and fuses it with salient target information to refine the saliency prediction, specifically addressing the issue of low confidence in boundary pixels.
  • Hybrid Loss Function: A combination of Binary Cross-Entropy (BCE) loss for pixel-wise accuracy and a Consistency Enhanced Loss (CEL) for image-level consistency is used. Additionally, a Saliency Boundary Detection Loss (SBDL) with a balanced binary entropy loss function is employed to handle the imbalance of boundary pixels during training.

Datasets

The authors evaluated their method on four publicly available datasets:

  • DUTS: A large dataset known for complex saliency target scenes.
  • DUT-OMRON: Features diverse image content with singular or plural salient objects.
  • HKU-IS: Characterized by multiple disconnected salient objects and inconspicuous contrast.
  • ECSSD: Contains rich semantics but complex structures.

Key Insights and Ideas

  • Boundary is Key: Explicitly modeling and enhancing object boundaries is crucial for improving the precision and robustness of saliency detection, especially in low-confidence regions.
  • Multi-Scale is Essential: Effectively capturing information at different scales is vital for handling the inherent scale variations of salient objects in real-world scenes.
  • Fusion for Refinement: Combining information from different feature levels and scales, along with boundary cues, leads to a more comprehensive and accurate saliency prediction.
  • Balanced Training: Using a hybrid loss function that considers both pixel-level accuracy and image-level consistency, alongside a balanced boundary loss, leads to more robust and uniform saliency maps.

Key Results

The proposed method demonstrates significant improvements over existing state-of-the-art methods:

  • Performance Improvement: On the DUTS dataset, the method achieved an average improvement of 6.2% in Mean Absolute Error (MAE) compared to the best existing method.
  • Comprehensive Evaluation: The method consistently outperformed others across various metrics (MAE, Fmax, Em, Sm) on the four benchmark datasets, using both VGG-16 and ResNet-50 backbones.
  • Qualitative Results: Visualizations show that the method not only highlights complete salient regions but also detects clear boundaries and ensures uniform pixel intensity within the predicted salient regions, even for complex scenes and small objects.

Limitations and Future Work

  • Limitations:
    • The performance of the boundary extraction module is critical; inaccurate boundary information can lead to suboptimal saliency maps.
  • The method might be sensitive to noise in input data, potentially causing false positives or negatives.
  • The computational complexity of the multi-stage approach could be a challenge for real-time applications on resource-constrained devices.
  • Future Work:
    • End-to-End Learning: Exploring end-to-end learning to streamline the pipeline and potentially reduce complexity.
  • Attention Mechanisms: Integrating attention mechanisms to improve focus on salient regions.
  • Saliency Propagation: Developing techniques to propagate saliency information through hierarchical architectures for better localization of objects at different scales.
  • Video and 3D Data: Extending the method to video and 3D point cloud data for applications like video object segmentation and 3D scene understanding.
  • Optical Scenarios: Adapting the method for specific challenges in optical wavefront analysis, such as Laguerre-Gaussian beams and turbulent atmospheres.

Summary updated

BibTeX
@misc{wen2023salient,
  author = {Falin Wen and Qinghui Wang and Ruirui Zou and Ying Wang and Fenglin Liu and Yang Chen and Linghao Yu and Shaoyi Du and Chengzhi Yuan},
  title = {{A Salient Object Detection Method Based on Boundary Enhancement}},
  year = {2023},
  doi = {10.3390/s23167077},
}
Figure