Saliency Transfer: An Example-Based Method for Salient Object Detection

Xin Li, Fan Yang, Leiting Chen, Hongbin CaiIJCAI 2016

Abstract

Salient Object Detection (SOD) is a fundamental task in computer vision aimed at identifying visually dominant regions in images. While traditional methods rely on heuristic intrinsic cues and modern deep learning models require extensive training data, this paper introduces Saliency Transfer (STR). STR is a training-free, example-based framework that "borrows" knowledge from a single, similar annotated image. By establishing dense pixel-wise correspondences and employing a coarse-to-fine optimization strategy, STR achieves state-of-the-art performance across multiple benchmarks, demonstrating that high-quality saliency detection is possible without large-scale training.

Introduction

The goal of Salient Object Detection is to pinpoint the most "important" objects in a scene—those that naturally draw human attention. Historically, researchers have utilized intrinsic cues, such as color uniqueness or spatial priors (e.g., objects are rarely at the image boundaries). However, these heuristics often struggle with complex backgrounds. Conversely, extrinsic cues derived from large-scale datasets offer better robustness but come with the heavy cost of massive training data and long computation times.

Saliency Transfer (STR) addresses this gap by proposing an example-based approach. The core insight is that if a similar image with a known ground-truth mask can be identified, its saliency information can be transferred and refined for the new input. This method allows for high-accuracy detection using a reference set of as few as 14 images, bypassing the need for "big data" training.

Methodology

The STR pipeline is divided into two primary stages: initial map generation via transfer and subsequent coarse-to-fine refinement.

Overview of the Saliency Transfer (STR) pipeline.

1. Initial Map Generation

The process begins by constructing a Saliency Feature (SF) image. For every pixel, a 10-dimensional vector is calculated, incorporating low-level cues like surroundedness (via Boolean maps), boundary priors, and convex hull priors. To find a suitable "buddy" image, a 1200-dimensional Global Saliency Feature (GSF) descriptor—a histogram summarizing the SF image—is used to retrieve the nearest neighbor from a small reference set.

Once a reference image is selected, an Annotation Transfer is performed. Using a dense flow field (similar to SIFT flow), the ground-truth mask of the reference image is warped onto the input image. This provides an initial, albeit rough, estimation of the salient object's location.

2. Coarse-to-Fine Optimization

The initial transferred mask is often misaligned or lacks sharp boundaries. To address this, the authors implement a multi-layer optimization framework. The image is recursively divided into grids, ranging from coarse blocks to individual pixels. At each level, Gaussian Mixture Models (GMMs) are used to model the color distributions of the foreground and background. This iterative refinement ensures the final saliency map is spatially smooth and precisely aligned with the actual object boundaries.

Experimental Evaluation

The STR method was rigorously tested on six popular benchmarks: MSRA1000, ECSSD, SED1, SED2, SOD, and iCoseg. These datasets range from simple single-object scenes to highly challenging images with multiple objects and cluttered backgrounds.

Qualitative comparison of STR against state-of-the-art methods and ground truth.

Key Results

STR consistently outperformed 11 leading algorithms, including RBD, DRFI, and HS. Quantitative highlights include:

  • Accuracy: On the ECSSD dataset, STR improved the weighted F-measure ($F_\beta^w$) by 25.16% over previous leading results.
  • Error Reduction: The method achieved a 39.05% reduction in Mean Absolute Error (MAE) on the MSRA1000 dataset.
  • Qualitative Superiority: As illustrated in comparative figures, STR produces "cleaner" maps with sharp boundaries, effectively avoiding the "ghosting" effects and background noise prevalent in heuristic models like DRFI.

Conclusion

Saliency Transfer demonstrates that a training-free, example-based approach can rival and even surpass complex data-driven models. Its ability to perform cross-category transfer—matching images based on saliency structure rather than semantic labels—makes it a versatile tool. While the current implementation takes approximately 6.45 seconds per image, future optimizations using GPU parallelism and automated reference set expansion could further enhance its utility for real-time applications.

Summary updated

BibTeX
@inproceedings{li2016saliency,
  author = {Xin Li and Fan Yang and Leiting Chen and Hongbin Cai},
  title = {{Saliency Transfer: An Example-Based Method for Salient Object Detection}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2016},
}
Figure