Abstract
Salient Object Detection (SOD) is a fundamental task in computer vision aimed at identifying the most visually prominent objects in an image. While RGB-based models have seen significant progress, they often struggle in complex environments with low contrast or cluttered backgrounds. This paper introduces DMRANet, a novel deep learning architecture that effectively integrates depth information (RGB-D) to provide structural cues. By mimicking the iterative refinement process of human visual attention, DMRANet employs a recurrent attention mechanism and depth-guided multi-scale weighting to achieve state-of-the-art performance across seven benchmark datasets.
Introduction
Traditional SOD models frequently fail in scenarios where objects are camouflaged against their backgrounds or in low-light conditions. Depth information offers a solution by providing geometric and structural context that is invariant to color and texture. However, existing RGB-D methods often suffer from two primary limitations: naive fusion strategies that ignore the multi-level relationship between RGB and depth data, and static, single-pass processing that lacks the refinement capabilities of the human brain's Internal Generative Mechanism (IGM).
DMRANet addresses these gaps through a two-stream architecture that treats depth as a structural guide rather than just an additional feature channel. The model introduces three key components: the Depth Refinement Block (DRB), the Depth-induced Multi-scale Weighting (DMSW) module, and the Recurrent Attention Module (RAM).
Proposed Methodology
The DMRANet architecture is built upon a VGG-19 backbone and processes RGB and depth inputs through parallel streams. The integration is handled by three specialized modules:
1. Depth Refinement Block (DRB)
Instead of simple concatenation, the DRB uses residual connections to learn depth-specific cues. It extracts complementary information from both streams at multiple levels, ensuring that high-level semantic information and low-level spatial details are preserved and fused effectively.
2. Depth-induced Multi-scale Weighting (DMSW)
Recognizing that objects appear at different scales depending on their distance from the camera, the DMSW module uses the depth map to generate a "depth vector." This vector acts as a scale factor to weight multi-scale context features generated via dilated convolutions. This allows the network to dynamically adjust its focus based on the spatial depth of the scene.
3. Recurrent Attention Module (RAM)
Inspired by the human brain's iterative "look and refine" process, the RAM utilizes a ConvLSTM to progressively optimize the saliency map. By maintaining a memory of previous iterations, the module can suppress background noise and sharpen object boundaries over multiple time steps.
Datasets and Evaluation
The model was evaluated on six public datasets: NJUD, NLPR, LFSD, STEREO, RGBD135, and SSD. Additionally, the authors introduced a new large-scale RGB-D dataset containing 1,200 paired images. This dataset is specifically designed to be more challenging, featuring transparent objects, multiple salient targets, and highly cluttered backgrounds.
Experimental Results
Quantitative Performance
DMRANet outperformed 16 state-of-the-art methods, ranking first across all seven datasets. On the challenging NJUD dataset, it achieved an F-measure of 0.872, significantly surpassing previous top-performing models like PCA and PDNet. It also maintained the lowest Mean Absolute Error (MAE) across the board, indicating superior precision in boundary localization.
Qualitative Analysis
Visual results demonstrate the model's robustness. While competing models often produce blurry maps or fail to detect objects with similar colors to the background, DMRANet successfully isolates complex targets, such as transparent glass bottles and small objects in dark rooms. The recurrent loops in the RAM are particularly effective at eliminating background "irritants" that confuse single-pass models.
Conclusion
DMRANet represents a significant advancement in RGB-D saliency detection by moving beyond naive fusion to a depth-guided, iterative refinement approach. The introduction of the DMSW and RAM modules allows the network to leverage depth as a structural guide for scale and attention. Future work may focus on optimizing the computational efficiency of the recurrent modules and improving robustness to noisy depth data from low-end sensors.
Summary updated