Abstract
Salient Object Detection (SOD) is a fundamental task in computer vision aimed at identifying visually dominant regions in images. While traditional methods rely on heuristic intrinsic cues and modern deep learning models require extensive training data, this paper introduces Saliency Transfer (STR). STR is a training-free, example-based framework that "borrows" knowledge from a single, similar annotated image. By establishing dense pixel-wise correspondences and employing a coarse-to-fine optimization strategy, STR achieves state-of-the-art performance across multiple benchmarks, demonstrating that high-quality saliency detection is possible without large-scale training.
Introduction
The goal of Salient Object Detection is to pinpoint the most "important" objects in a scene—those that naturally draw human attention. Historically, researchers have utilized intrinsic cues, such as color uniqueness or spatial priors (e.g., objects are rarely at the image boundaries). However, these heuristics often struggle with complex backgrounds. Conversely, extrinsic cues derived from large-scale datasets offer better robustness but come with the heavy cost of massive training data and long computation times.
Saliency Transfer (STR) addresses this gap by proposing an example-based approach. The core insight is that if a similar image with a known ground-truth mask can be identified, its saliency information can be transferred and refined for the new input. This method allows for high-accuracy detection using a reference set of as few as 14 images, bypassing the need for "big data" training.
Methodology
The STR pipeline is divided into two primary stages: initial map generation via transfer and subsequent coarse-to-fine refinement.
1. Initial Map Generation
The process begins by constructing a Saliency Feature (SF) image. For every pixel, a 10-dimensional vector is calculated, incorporating low-level cues like surroundedness (via Boolean maps), boundary priors, and convex hull priors. To find a suitable "buddy" image, a 1200-dimensional Global Saliency Feature (GSF) descriptor—a histogram summarizing the SF image—is used to retrieve the nearest neighbor from a small reference set.
Once a reference image is selected, an Annotation Transfer is performed. Using a dense flow field (similar to SIFT flow), the ground-truth mask of the reference image is warped onto the input image. This provides an initial, albeit rough, estimation of the salient object's location.
2. Coarse-to-Fine Optimization
The initial transferred mask is often misaligned or lacks sharp boundaries. To address this, the authors implement a multi-layer optimization framework. The image is recursively divided into grids, ranging from coarse blocks to individual pixels. At each level, Gaussian Mixture Models (GMMs) are used to model the color distributions of the foreground and background. This iterative refinement ensures the final saliency map is spatially smooth and precisely aligned with the actual object boundaries.
Experimental Evaluation
The STR method was rigorously tested on six popular benchmarks: MSRA1000, ECSSD, SED1, SED2, SOD, and iCoseg. These datasets range from simple single-object scenes to highly challenging images with multiple objects and cluttered backgrounds.
Key Results
STR consistently outperformed 11 leading algorithms, including RBD, DRFI, and HS. Quantitative highlights include:
- Accuracy: On the ECSSD dataset, STR improved the weighted F-measure ($F_\beta^w$) by 25.16% over previous leading results.
- Error Reduction: The method achieved a 39.05% reduction in Mean Absolute Error (MAE) on the MSRA1000 dataset.
- Qualitative Superiority: As illustrated in comparative figures, STR produces "cleaner" maps with sharp boundaries, effectively avoiding the "ghosting" effects and background noise prevalent in heuristic models like DRFI.
Conclusion
Saliency Transfer demonstrates that a training-free, example-based approach can rival and even surpass complex data-driven models. Its ability to perform cross-category transfer—matching images based on saliency structure rather than semantic labels—makes it a versatile tool. While the current implementation takes approximately 6.45 seconds per image, future optimizations using GPU parallelism and automated reference set expansion could further enhance its utility for real-time applications.
Summary updated