Supervision by Fusion: Towards Unsupervised Learning of Deep Salient Object Detector

Dingwen Zhang, Junwei Han, Yu ZhangICCV 2017 · OpenReview

Abstract

Salient Object Detection (SOD) has seen significant advancements through Deep Neural Networks (DNNs), yet these models typically rely on extensive, labor-intensive pixel-level human annotations. This paper introduces a novel "supervision by fusion" framework that enables the training of deep salient object detectors entirely without manual labels. By leveraging a collective of weak, unsupervised saliency models to generate pseudo-ground truth and a dynamic learning curriculum, the proposed method achieves performance that rivals, and in some cases exceeds, fully supervised state-of-the-art methods.

The Challenge of Data-Hungry Saliency Detection

SOD aims to identify the most visually prominent regions in an image. While DNNs have revolutionized the field, their reliance on thousands of precisely annotated images presents a significant bottleneck. Conversely, traditional unsupervised methods based on hand-crafted features (like color contrast or boundary priors) are computationally inexpensive but lack the robustness and accuracy of deep models. The authors address this by asking: Can we build a powerful deep saliency detector using only raw image data, bypassing the need for human supervision entirely?

Methodology: Supervision by Fusion

The core innovation is the Supervision by Fusion strategy, where the network learns from the "collective wisdom" of multiple weak unsupervised models rather than human labels.

1. Weak Model Collection and Fusion

The framework begins with three fast, unsupervised detectors (e.g., MBS, BMS, and HS). These are combined through a two-stream fusion process:

  • Intra-image Fusion: Operates at the superpixel level within a single image, using a probabilistic model (GLAD) to determine which superpixels are "easy" or "hard" and which weak detectors are most reliable for that specific context.
  • Inter-image Fusion: Operates across the entire dataset to identify globally "easier" images and assign reliability weights to detectors based on their aggregate performance.

2. Pseudo Ground-Truth and Curriculum Learning

This fusion produces a pseudo-saliency map (the training target) and a Learning Confidence Map (LCM). The LCM serves as a curriculum, guiding the network to prioritize high-confidence, consistent regions while down-weighting noisy or ambiguous samples during the early stages of training.

The framework utilizes a deep architecture (DHSNet) trained on pseudo-labels generated from raw images.

3. Iterative Refinement

The network, based on the DHSNet architecture, is trained using these fused signals. In a self-improving loop, the deep model's own predictions are subsequently used to update and refine the weak saliency collection in later iterations.

Experimental Results

The model was trained on the MSRA10K dataset (with human labels discarded) and evaluated on four standard benchmarks: ECSSD, SOD, DUT-O, and PASCAL-S.

Performance and Efficiency

  • Closing the Gap: The unsupervised model performs within 2-5% of the exact same architecture trained with full human supervision.
  • Outperforming Supervised Baselines: Remarkably, the method outperformed several older fully supervised state-of-the-art deep models, such as LEGS, MDF, and MCDL.
  • Resource Efficiency: While manual annotation takes approximately 31 seconds per image, generating the weak maps for this framework requires only 0.7 seconds per image.
Quantitative and qualitative comparison showing the performance gap between traditional and deep detectors across multiple datasets.

Key Insights and Future Directions

The success of this approach lies in treating the inconsistency between unsupervised models as a valuable signal. By using an "external force" (the fusion of independent models) to build the curriculum, the framework prevents the model from reinforcing its own early errors, a common pitfall in standard self-paced learning.

While a small performance gap remains compared to full supervision—limited by the quality of the initial weak models—the "supervision by fusion" concept offers a promising path for other data-starved computer vision tasks, including co-saliency detection, semantic segmentation, and object parsing.

Summary updated

BibTeX
@inproceedings{zhang2017supervision,
  author = {Dingwen Zhang and Junwei Han and Yu Zhang},
  title = {{Supervision by Fusion: Towards Unsupervised Learning of Deep Salient Object Detector}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2017},
  url = {https://openreview.net/forum?id=prhX3KqpYob},
}
Figure