Abstract
Unsupervised Salient Object Detection (USOD) aims to identify significant objects without manual labels. However, existing methods suffer from error accumulation and limited domain generalization. We present a unified framework that addresses these challenges through a two-stage process: Progressive Curriculum Learning-based Saliency Distilling (PCL-SD) and Self-rectify Pseudo-label Refinement (SPR). By leveraging a curriculum that transitions from easy to hard samples, we mitigate noisy label propagation. Furthermore, we introduce an adapter-tuning mechanism to efficiently transfer knowledge from natural still images to specialized domains like thermal, video, and remote sensing. Experimental results across five SOD tasks demonstrate state-of-the-art performance, highlighting the framework's robustness and versatility.
Introduction
Salient Object Detection (SOD) is a fundamental computer vision task focused on identifying the most visually prominent objects in an image. While supervised deep learning has pushed the boundaries of accuracy, the reliance on massive pixel-level annotations remains a bottleneck, particularly for niche domains such as thermal (RGB-T) or satellite imagery (RSI). Unsupervised methods (USOD) offer a promising alternative but are often plagued by error accumulation—where initial mistakes on complex "hard" samples propagate through training—and a lack of cross-domain generalization.
Methodology
The proposed framework operates in two distinct stages to ensure stable learning and high-quality saliency map generation.
Stage 1: Progressive Curriculum Learning-based Saliency Distilling (PCL-SD)
To combat error accumulation, the model employs a curriculum learning strategy. Initially, the Saliency Cue Extractor (SCE) is trained only on "easy" pixels—those where the pre-trained network exhibits high confidence. As the model matures, the difficulty threshold is gradually lowered, allowing it to learn from increasingly complex spatial contexts and boundaries. This "easy-to-hard" progression prevents the model from collapsing due to noisy pseudo-labels early in the training phase.
Stage 2: Self-rectify Pseudo-label Refinement (SPR)
In the second stage, a Saliency Detector (SD) is trained using the refined cues from Stage 1. To further enhance the quality of these pseudo-labels, the authors introduce SPR. This mechanism fuses the model's current predictions with a spatial prior derived from the raw image's low-level features (e.g., color and position). This real-time refinement helps restore fine-grained local details that are often blurred in deep feature maps.
Cross-Domain Knowledge Transfer
A key innovation is the use of Adapter-tuning to generalize the model to specialized domains like Video SOD or RSI SOD. Recognizing that shallow features are often universal while deep features are task-specific, the framework freezes the pre-trained backbone and inserts small, learnable "adapter" modules into the deeper layers. This approach enables effective domain adaptation without the risk of catastrophic forgetting or model degradation.
Experimental Results
The framework was evaluated across five distinct SOD tasks, including RGB, RGB-D, RGB-T, Video, and Remote Sensing.
- State-of-the-Art Performance: On the RGBD-135 dataset, the model achieved a Mean Absolute Error (MAE) of 0.025, outperforming the previous unsupervised SOTA (TSD, 0.029).
- Zero-Shot and Fine-Tuning: The model demonstrated strong zero-shot capabilities. In the RSI SOD task (EORSSD dataset), the F-measure improved from 0.545 (zero-shot) to 0.625 after adapter-tuning, validating the efficiency of the transfer mechanism.
Conclusion
This work presents a robust, unified framework for unsupervised salient object detection that effectively bridges the gap between natural still images and specialized domains. By combining curriculum-based distilling with efficient adapter-tuning, the method achieves high accuracy without requiring manual annotations, paving the way for scalable saliency detection in diverse real-world applications.
Summary updated