Abstract
Salient object detection is a fundamental computer vision task. The majority of existing algorithms focus on aggregating multi-level features of pre-trained convolutional neural networks. Moreover, some researchers attempt to utilize edge information for auxiliary training. However, existing edge-aware models design unidirectional frameworks which only use edge features to improve the segmentation features. Motivated by the logical interrelations between binary segmentation and edge maps, we propose a novel Stacked Cross Refinement Network (SCRN) for salient object detection in this paper. Our framework aims to simultaneously refine multi-level features of salient object detection and edge detection by stacking Cross Refinement Unit (CRU). According to the logical interrelations, the CRU designs two direction-specific integration operations, and bidirectionally passes messages between the two tasks. Incorporating the refined edge-preserving features with the typical U-Net, our model detects salient objects accurately. Extensive experiments conducted on six benchmark datasets demonstrate that our method outperforms existing state-of-the-art algorithms in both accuracy and efficiency. Besides, the attribute-based performance on the SOC dataset show that the proposed model ranks first in the majority of challenging scenes.
Introduction
Salient Object Detection (SOD) aims to identify and segment the most visually significant objects in an image. A persistent challenge in SOD is the generation of clean, sharp boundaries; many existing models produce blurry heatmaps that bleed into the background. While edge information has been used to assist segmentation, previous approaches typically employed unidirectional frameworks where edge features improved the object map, but not vice versa.
The Stacked Cross Refinement Network (SCRN) addresses this by treating edge detection and object segmentation as mutually beneficial, bidirectional tasks. This approach is grounded in the logical link that an object's boundary is a subset of its total area. Consequently, spatial information from the object region can help localize edges, while precise edge information can sharpen the object's boundaries.
Methodology
The SCRN architecture utilizes a ResNet-50 backbone to extract multi-level features, which are then processed through two parallel decoders for edge and salient object detection.
Cross Refinement Unit (CRU)
The core of the SCRN is the Cross Refinement Unit (CRU), which facilitates bidirectional message passing between the two branches:
- Segmentation to Edge ($S \to E$): Employs element-wise multiplication to ensure edge features are primarily active within the predicted object region.
- Edge to Segmentation ($S \leftarrow E$): Uses concatenation to integrate edge information, helping to "fill in" and sharpen the object's boundaries.
Iterative Refinement via Stacking
To progressively clean and sharpen features, the model stacks four CRUs in sequence. This iterative process allows the network to re-evaluate features multiple times, effectively increasing the receptive field and suppressing background noise. A "Selective Set-to-Point Style" (SSPS) strategy is further employed within the CRUs to aggregate higher-level features while ignoring lower-level noise that might contain distracting background information.
Experimental Results
The SCRN was trained on the DUTS-TR dataset and evaluated across six standard benchmarks: ECSSD, PASCAL-S, DUTS-TE, HKU-IS, DUT-OMRON, and THUR15K.
Performance and Efficiency
As shown in the quantitative results, SCRN consistently outperforms ten state-of-the-art algorithms, achieving the lowest Mean Absolute Error (MAE) and highest F-measure across all tested datasets. Notably, the model maintains high efficiency, running at 52 frames per second (FPS) on a single Titan X GPU, making it significantly faster than many competing high-accuracy models.
Robustness in Complex Scenes
Evaluations on the SOC (Salient Objects in Clutter) dataset further demonstrate the model's robustness. SCRN ranked first in the majority of challenging categories, such as low contrast and large objects, proving its capability in handling complex real-world scenarios where objects are partially occluded or blend into the background.
Conclusion
The Stacked Cross Refinement Network (SCRN) introduces a novel bidirectional refinement strategy for salient object detection. By leveraging the logical interrelations between edges and object masks through stacked CRUs, the model achieves state-of-the-art performance with high computational efficiency. This bidirectional approach provides a promising framework for other related vision tasks, such as semantic segmentation and instance boundary detection.
Summary updated