Multi-Source Uncertainty Mining for Deep Unsupervised Saliency Detection

Yifan Wang, Wenbo Zhang, Lijun Wang, Ting Liu, Huchuan Lu

Introduction

Salient Object Detection (SOD) is a fundamental computer vision task aimed at identifying the most visually prominent objects in an image. While deep learning has significantly advanced SOD performance, these models typically rely on vast quantities of pixel-level human annotations. Producing such labels is labor-intensive and expensive, creating a bottleneck for scaling these technologies.

To circumvent the need for manual labeling, researchers have explored unsupervised SOD, which leverages traditional, handcrafted algorithms (e.g., those based on color contrast or boundary priors) to generate "pseudo-labels." However, these handcrafted methods often produce noisy or inconsistent results. Training a deep network directly on this flawed data often leads the model to simply replicate the errors of the source algorithms.

The UMNet Framework

This paper introduces UMNet, a novel framework designed to "mine" reliability from multiple noisy sources. The core challenge addressed is: how can a network learn to distinguish between accurate and inaccurate pseudo-labels without ever seeing human-verified ground truth?

The authors propose a dual-network architecture consisting of a SODNet (the primary saliency detector) and an Uncertainty Mining Network (UMNet), which is specialized in estimating the reliability of each pseudo-label.

Multi-Source Consensus and Uncertainty

Instead of relying on a single noisy source, UMNet utilizes four different handcrafted SOD methods. The underlying intuition is that while any single method may fail, their collective agreement provides a strong signal for reliability. Areas where multiple methods agree are likely to be correct, whereas areas of high disagreement indicate uncertainty.

Visualizing the relationship between original images, noisy pseudo-labels, and the uncertainty maps generated to guide training.

As shown in the figure above, the framework takes an input image (a) and processes multiple pseudo-labels (b). The UMNet then generates uncertainty maps (d) that correspond to the deviations of these pseudo-labels from the latent "true" saliency. This allows the system to treat high-agreement areas as high-confidence training signals.

The Merge-and-Split (MS) Module

The technical heart of UMNet is the recursive Merge-and-Split (MS) module. These modules process image features alongside multiple pseudo-labels. The "Merge" operation identifies consensus across sources, while the "Split" operation analyzes how individual labels deviate from that consensus. This hierarchical processing allows the network to understand the spatial context—such as complex backgrounds—that might cause a specific handcrafted algorithm to fail.

Uncertainty-Weighted Learning

The training process employs an Uncertainty Weighted Loss derived from a Bayesian framework using the Gibbs distribution. This loss function dynamically adjusts the importance of pixels during training:

  • High Confidence: Pixels where UMNet reports low uncertainty are given high weight, forcing SODNet to learn from these reliable signals.
  • Low Confidence: Pixels with high uncertainty are effectively ignored, preventing the network from learning from noisy or incorrect labels.

Crucially, UMNet and SODNet are trained jointly. As SODNet improves at detecting objects, it provides better feedback to UMNet, which in turn refines its uncertainty estimates, creating a virtuous cycle of improvement.

Performance and Results

The model was trained on the MSRA-B dataset using only noisy pseudo-labels and evaluated across five standard benchmarks, including DUTS and DUT-OMRON.

Key findings include:

  • State-of-the-Art Unsupervised Performance: On the DUTS dataset, UMNet achieved a Mean Absolute Error (MAE) of 0.0667, surpassing previous leading unsupervised methods like DeepUSPS.
  • Competitive with Supervised Models: Despite the lack of human labels, UMNet's performance in terms of F-measure and S-measure is comparable to established supervised models such as Amulet and DCL.
  • Ablation Success: The authors demonstrated that using multiple sources is critical; performance dropped significantly when the model was restricted to a single source of pseudo-labels.

Conclusion and Future Directions

UMNet demonstrates that deep networks can effectively learn complex tasks like saliency detection by intelligently mining consensus from noisy, traditional algorithms. By modeling uncertainty explicitly, the framework bridges the gap between unsupervised and supervised learning.

A remaining challenge is the "Consensus Error"—situations where all handcrafted sources agree on an incorrect prediction (e.g., a high-contrast background element). Future research may focus on integrating self-supervised image priors or more diverse data sources to mitigate these collective failures.

Summary updated

Abstract

Deep learning-based image salient object detection (SOD) heavily relies on large-scale training data with pixel-wise labeling. High-quality labels involve intensive labor and are expensive to acquire. In this paper, we propose a novel multi-source uncertainty mining method to facilitate unsupervised deep learning from multiple noisy labels generated by traditional handcrafted SOD methods. We design an Uncertainty Mining Network (UMNet) which consists of multiple Merge-and-Split (MS) modules to recursively analyze the commonality and difference among multiple noisy labels and infer pixel-wise uncertainty map for each label. Meanwhile, we model the noisy labels using Gibbs distribution and propose a weighted uncertainty loss to jointly train the UMNet with the SOD network. As a consequence, our UMNet can adaptively select reliable labels for SOD network learning. Extensive experiments on benchmark datasets demonstrate that our method not only outperforms existing unsupervised methods, but also is on par with fully-supervised state-of-the-art models.
BibTeX
@inproceedings{wang2022multi,
  author = {Yifan Wang and Wenbo Zhang and Lijun Wang and Ting Liu and Huchuan Lu},
  title = {{Multi-Source Uncertainty Mining for Deep Unsupervised Saliency Detection}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2022},
  doi = {10.1109/cvpr52688.2022.01143},
}
Figure