Introduction
Salient Object Detection (SOD) is a fundamental computer vision task that involves identifying and segmenting the most visually prominent objects in an image. While Fully Convolutional Networks (FCNs) have significantly advanced the state-of-the-art, they often struggle with two primary issues: feature clutter and the loss of fine spatial details. Low-level features in deep networks frequently contain background noise that can mislead the segmentation process, while downsampling operations like pooling inevitably blur object boundaries.
This paper introduces the Deep Embedding Feature (DEF) network, a novel architecture designed to address these challenges. By embedding prior predictions into a metric space and utilizing a recursive refinement strategy with a learnable guided filter, the DEF network achieves high-accuracy saliency masks with crisp boundaries while maintaining real-time performance at 38 FPS.
Methodology
The DEF architecture is a multi-branch system designed to filter noise and recover spatial resolution progressively through four main components:
- Initial Saliency Prediction (ISP): Using a ResNet-101 backbone, the model first generates a coarse saliency map and extracts multi-scale feature maps from the input image.
- Feature Embedding Network (FEN): This core component addresses feature clutter. Instead of simple concatenation, FEN embeds coarse predictions and multi-level features into a metric space. This process calculates the spatial importance of each pixel, effectively filtering out background noise and keeping only "attentive" features focused on the salient object.
- Recursive Feature Integration Network (RFIN): To recover lost spatial details, RFIN takes the embedded features and integrates them with side-output features in a deep-to-shallow sequence. It generates stage-wise predictions, where each stage is supervised by the ground truth, allowing the network to progressively correct its own errors.
- Guided Filter Refinement Network (GFRN): To fix the blurry boundary problem, the authors use a learnable guided filter. It takes the original, high-resolution RGB image as a "guidance map" to snap the edges of the RFIN's predicted mask to the actual physical boundaries of the object.
Experiments and Results
The model was trained on the DUTS training dataset and benchmarked against five standard, large-scale public datasets: ECSSD, PASCAL-S, DUT-OMRON, DUTS-TE, and HKU-IS.
Quantitative Performance
The DEF network consistently outperformed 14 state-of-the-art models across all benchmarks. For example, it achieved an impressive F-measure of 0.915 on ECSSD and 0.907 on HKU-IS. Beyond accuracy, the model is highly efficient, achieving an inference speed of 38 Frames Per Second (FPS) on a single GTX 1080 Ti GPU, making it viable for real-time video processing.
Qualitative Analysis
Qualitatively, the DEF network excels in difficult scenarios such as low-contrast backgrounds or complex textures. As shown in the visual comparisons, the embedding process successfully isolates the salient object from background clutter. While standard FCNs often produce noisy or blurry results, the DEF network's recursive refinement and guided filtering ensure sharp, accurate boundaries that align closely with the ground truth.
Conclusion
The Deep Embedding Feature network demonstrates that intelligent feature filtering is just as critical as feature depth in salient object detection. By treating feature integration as a metric embedding problem and employing recursive refinement, the model overcomes the traditional limitations of FCNs. The authors suggest that this approach to noise suppression and boundary preservation is highly adaptable and could be effective for other low-level vision tasks, such as semantic segmentation and edge detection.
Summary updated