Deep Embedding Features for Salient Object Detection

Yun-Zhi Zhuge, Yu Zeng, Huchuan Lu

Introduction

Salient Object Detection (SOD) is a fundamental computer vision task that involves identifying and segmenting the most visually prominent objects in an image. While Fully Convolutional Networks (FCNs) have significantly advanced the state-of-the-art, they often struggle with two primary issues: feature clutter and the loss of fine spatial details. Low-level features in deep networks frequently contain background noise that can mislead the segmentation process, while downsampling operations like pooling inevitably blur object boundaries.

This paper introduces the Deep Embedding Feature (DEF) network, a novel architecture designed to address these challenges. By embedding prior predictions into a metric space and utilizing a recursive refinement strategy with a learnable guided filter, the DEF network achieves high-accuracy saliency masks with crisp boundaries while maintaining real-time performance at 38 FPS.

Methodology

The DEF architecture is a multi-branch system designed to filter noise and recover spatial resolution progressively through four main components:

  • Initial Saliency Prediction (ISP): Using a ResNet-101 backbone, the model first generates a coarse saliency map and extracts multi-scale feature maps from the input image.
  • Feature Embedding Network (FEN): This core component addresses feature clutter. Instead of simple concatenation, FEN embeds coarse predictions and multi-level features into a metric space. This process calculates the spatial importance of each pixel, effectively filtering out background noise and keeping only "attentive" features focused on the salient object.
  • Recursive Feature Integration Network (RFIN): To recover lost spatial details, RFIN takes the embedded features and integrates them with side-output features in a deep-to-shallow sequence. It generates stage-wise predictions, where each stage is supervised by the ground truth, allowing the network to progressively correct its own errors.
  • Guided Filter Refinement Network (GFRN): To fix the blurry boundary problem, the authors use a learnable guided filter. It takes the original, high-resolution RGB image as a "guidance map" to snap the edges of the RFIN's predicted mask to the actual physical boundaries of the object.
The architecture of the proposed DEF network, highlighting the embedding learning module (ELM) and integration stages.

Experiments and Results

The model was trained on the DUTS training dataset and benchmarked against five standard, large-scale public datasets: ECSSD, PASCAL-S, DUT-OMRON, DUTS-TE, and HKU-IS.

Quantitative Performance

The DEF network consistently outperformed 14 state-of-the-art models across all benchmarks. For example, it achieved an impressive F-measure of 0.915 on ECSSD and 0.907 on HKU-IS. Beyond accuracy, the model is highly efficient, achieving an inference speed of 38 Frames Per Second (FPS) on a single GTX 1080 Ti GPU, making it viable for real-time video processing.

Qualitative Analysis

Qualitatively, the DEF network excels in difficult scenarios such as low-contrast backgrounds or complex textures. As shown in the visual comparisons, the embedding process successfully isolates the salient object from background clutter. While standard FCNs often produce noisy or blurry results, the DEF network's recursive refinement and guided filtering ensure sharp, accurate boundaries that align closely with the ground truth.

Qualitative comparison showing (a) input image, (b-c) intermediate feature embeddings, (d) ground truth, and (e-f) saliency predictions demonstrating the noise suppression of the proposed method.

Conclusion

The Deep Embedding Feature network demonstrates that intelligent feature filtering is just as critical as feature depth in salient object detection. By treating feature integration as a metric embedding problem and employing recursive refinement, the model overcomes the traditional limitations of FCNs. The authors suggest that this approach to noise suppression and boundary preservation is highly adaptable and could be effective for other low-level vision tasks, such as semantic segmentation and edge detection.

Summary updated

Abstract

Benefiting from the rapid development of Convolutional Neural Networks (CNNs), some salient object detection methods have achieved remarkable results by utilizing multi-level convolutional features. However, the saliency training datasets is of limited scale due to the high cost of pixel-level labeling, which leads to a limited generalization of the trained model on new scenarios during testing. Besides, some FCN-based methods directly integrate multi-level features, ignoring the fact that the noise in some features are harmful to saliency detection. In this paper, we propose a novel approach that transforms prior information into an embedding space to select attentive features and filter out outliers for salient object detection. Our network firstly generates a coarse prediction map through an encorder-decorder structure. Then a Feature Embedding Network (FEN) is trained to embed each pixel of the coarse map into a metric space, which incorporates much attentive features that highlight salient regions and suppress the response of non-salient regions. Further, the embedded features are refined through a deep-to-shallow Recursive Feature Integration Network (RFIN) to improve the details of prediction maps. Moreover, to alleviate the blurred boundaries, we propose a Guided Filter Refinement Network (GFRN) to jointly optimize the predicted results and the learnable guidance maps. Extensive experiments on five benchmark datasets demonstrate that our method outperforms state-of-the-art results. Our proposed method is end-to-end and achieves a realtime speed of 38 FPS.
BibTeX
@inproceedings{zhuge2019deep,
  author = {Yun-Zhi Zhuge and Yu Zeng and Huchuan Lu},
  title = {{Deep Embedding Features for Salient Object Detection}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2019},
  doi = {10.1609/aaai.v33i01.33019340},
}
Figure