Abstract
Recently, unsupervised salient object detection (USOD) has gained increasing attention due to its annotation-free nature. However, current methods mainly focus on specific tasks such as RGB and RGB-D, neglecting the potential for task migration. In this paper, we propose a unified USOD framework for generic USOD tasks. Firstly, we propose a Progressive Curriculum Learning-based Saliency Distilling (PCL-SD) mechanism to extract saliency cues from a pre-trained deep network. This mechanism starts with easy samples and progressively moves towards harder ones, to avoid initial interference caused by hard samples. Afterwards, the obtained saliency cues are utilized to train a saliency detector, and we employ a Self-rectify Pseudo-label Refinement (SPR) mechanism to improve the quality of pseudo-labels. Finally, an adapter-tuning method is devised to transfer the acquired saliency knowledge, leveraging shared knowledge to attain superior transferring performance on the target tasks. Extensive experiments on five representative SOD tasks confirm the effectiveness and feasibility of our proposed method.
Introduction
Salient Object Detection (SOD) is the task of identifying the most visually important objects in a scene. While supervised methods are highly accurate, they require massive amounts of expensive, pixel-level manual annotations. Unsupervised methods (USOD) avoid this but face two major hurdles: the "Hard Sample Trap," where early training on complex scenes leads to error accumulation, and "Domain Silos," where research is often restricted to standard RGB photos without clear paths to specialized domains like Video or Remote Sensing.
To overcome these challenges, we introduce a unified framework for USOD that moves beyond specific image types to handle RGB, depth, thermal, video, and remote sensing data. By combining curriculum learning to handle hard samples and a novel adapter-tuning strategy, we successfully transfer saliency knowledge across domains without requiring a single manual pixel-level annotation.
Methodology
The proposed framework is centered on a two-stage knowledge transfer process designed to distill and refine saliency information.
Stage 1: Progressive Curriculum Learning-based Saliency Distilling (PCL-SD)
Instead of training on all data simultaneously, we utilize a curriculum approach. The model starts by learning from "easy" pixels—those where the pre-trained network is most certain. As training progresses, the threshold for "hard" samples is lowered. This prevents the model from being overwhelmed by noise early on, building a stable foundation for identifying complex object boundaries.
Stage 2: Saliency Detector Training and Pseudo-label Refinement
The distilled cues are used to train a final saliency detector via pseudo-labels. To improve label quality, we employ a Self-rectify Pseudo-label Refinement (SPR) mechanism. SPR combines the model's own predictions with a pixel refiner that considers raw image colors and positions, preventing the model from ignoring fine details or becoming "self-complacent."
Unified Transfer via Adapter-Tuning
To move from standard photos to specialized domains like video or satellite imagery, we keep the majority of the model frozen and only fine-tune small "adapter" modules. This allows the model to adapt to new domain-specific features (like motion) while retaining the core "objectness" knowledge learned from natural still images.
Experimental Evaluation
The study evaluated the model across five distinct SOD tasks, including RGB, RGB-D (Depth), RGB-T (Thermal), Video, and Remote Sensing (RSI).
Key results include:
- State-of-the-Art Performance: The proposed method consistently outperformed existing USOD methods across all five benchmarks.
- Closing the Supervision Gap: In many cases, this unsupervised method performed competitively with weakly-supervised and even some fully-supervised models. For example, on the ECSSD dataset, it achieved an F-measure of .923.
- Cross-Domain Synergy: Results suggest that training on a mix of data types actually improves performance on any single task, proving that "saliency" is a shared concept across different sensor types.
Conclusion
This paper presents a unified framework that effectively bridges the gap between different USOD tasks. By leveraging curriculum learning and adapter-tuning, we demonstrate that knowledge from natural images can serve as a powerful starting point for specialized domains. Future work will explore more complex adapters and investigate transfer capabilities in even more diverse environments, such as medical or underwater imaging.
Summary updated