Abstract
Detecting and segmenting salient objects from given image scenes has received great attention in recent years. A fundamental challenge in training the existing deep saliency detection models is the requirement of large amounts of annotated data. While gathering large quantities of training data becomes cheap and easy, annotating the data is an expensive process in terms of time, labor and human expertise. To address this problem, this paper proposes to learn the effective salient object detection model based on the manual annotation on a few training images only, thus dramatically alleviating human labor in training models. To this end, we name this task as the few-cost salient object detection and propose an adversarial-paced learning (APL)-based framework to facilitate the few-cost learning scenario. Essentially, APL is derived from the self-paced learning (SPL) regime but it infers the robust learning pace through the data-driven adversarial learning mechanism rather than the heuristic design of the learning regularizer. Comprehensive experiments on four widely-used benchmark datasets demonstrate that the proposed method can effectively approach to the existing supervised deep salient object detection models with only 1k human-annotated training images.
Introduction
Salient Object Detection (SOD)—the task of identifying and segmenting the most "eye-catching" objects in an image—typically relies on deep learning models trained on massive datasets. However, providing pixel-wise ground-truth masks for thousands of images is incredibly expensive and time-consuming. While unsupervised or weakly supervised methods exist, they often struggle with accuracy.
This research identifies a middle ground: Few-Cost SOD. In this scenario, a large pool of images is available, but only a small fraction (e.g., 1,000 out of 10,000) have manual annotations. The challenge lies in "mining" knowledge from the unlabeled images without letting the inevitable errors in predicted "pseudo-labels" degrade the model's performance.
Methodology: Adversarial-Paced Learning
The core innovation is Adversarial-Paced Learning (APL). APL evolves from two established paradigms:
- Self-Paced Learning (SPL): A regime where a model starts with "easy" samples and gradually includes "harder" ones based on a human-designed mathematical formula (a regularizer).
- Generative Adversarial Learning (GAL): A framework where a "generator" tries to create realistic data and a "discriminator" tries to distinguish between real and fake.
Instead of using a hard-coded formula to decide which unlabeled samples are "reliable," the authors use a Pace-Generator. This component acts like a discriminator: it learns to distinguish between high-quality manual annotations and low-quality predicted masks.
As shown in the architecture diagram, the Pace-Generator consists of a Global Structure Mining (GSM) branch to judge the overall "realness" of a shape and a Pixel Weighting (PW) branch to assign reliability scores to specific pixels, such as tricky object boundaries. By training the Task-Predictor and Pace-Generator in an adversarial loop, the model automatically learns a "robust learning pace," focusing on the most reliable parts of the unlabeled data as training progresses.
Experiments and Results
The models were evaluated on standard benchmarks including DUTS-TE, PASCAL-S, DUT-O, and SOD. The training split of DUTS (approx. 10k images) was used, but manual annotations were restricted to only 1,000 images to simulate the few-cost scenario.
Qualitative Reliability
The effectiveness of the APL framework is visible in how it assigns weights to unlabeled data. Compared to traditional SPL or standard GAN-based weighting, the proposed method produces more refined weight maps that better capture the reliability of predicted masks.
Quantitative Performance
- Efficiency: Using only 1,000 annotated images, the APL-based model outperformed all existing unsupervised, semi-supervised, and weakly supervised methods.
- Performance Gap: On the DUTS-TE dataset, the proposed method achieved a Max F-measure of 0.846, coming remarkably close to fully supervised models (like PoolNet at 0.894) that used ten times more annotated data.
- Robustness: Even when the ratio of labeled data was dropped to just 1% (approx. 100 images), the model still achieved 95.48% of the performance of a model trained on the full dataset.
Conclusion
Adversarial-Paced Learning demonstrates that data-driven mechanisms for determining learning pace are superior to heuristic designs. By combining global structure analysis with local pixel weighting, the APL framework allows a small "seed" of high-quality data to act as a powerful anchor for a much larger unlabeled dataset, significantly reducing the human cost of training high-performance saliency models.
Summary updated