Cosaliency Detection Based on Intrasaliency Prior Transfer and Deep Intersaliency Mining

Dingwen Zhang, Junwei Han, Jungong Han, Ling Shao

Abstract

Cosaliency detection aims to identify common salient objects within a group of related images. This paper introduces a novel framework that utilizes Stacked Denoising Autoencoders (SDAEs) to address the challenges of varying object appearances and limited training data. The approach consists of two main components: transferring intrasaliency priors from large auxiliary datasets and mining deep intersaliency patterns via unsupervised reconstruction residuals. Experimental results on standard benchmarks demonstrate that this method significantly outperforms traditional shallow-feature-based approaches.

Introduction

In computer vision, saliency detection identifies regions that naturally attract human attention. Cosaliency detection extends this to image groups, requiring the identification of objects that are both salient within their respective images (intrasaliency) and common across the entire set (intersaliency). Traditional methods often struggle when common objects exhibit significant variations in pose, lighting, or background. Zhang et al. (2016) proposed a deep-learning-based solution to capture these complex semantic commonalities.

Methodology

The proposed framework follows a three-stage pipeline to generate final cosaliency maps, integrating both intra-image and inter-image cues.

The proposed cosaliency detection framework, illustrating the flow from input image groups to final co-saliency maps via intrasaliency transfer and intersaliency discovery.

1. Intrasaliency Prior Transfer

To avoid the limitations of hand-crafted rules, the authors employ transfer learning. They train an SDAE on a large auxiliary dataset (ASD) to learn abstract "contrast" priors—measuring how distinct a region is from image boundaries. This "transfer machine" allows the model to predict saliency in new image groups without needing group-specific labels.

2. Deep Intersaliency Mining

The core innovation is the use of SDAEs for unsupervised pattern mining. The model takes salient regions from all images in a group to self-train a new SDAE. Because the network learns to represent the group's commonalities, regions that fit these patterns yield low reconstruction residuals. These residuals serve as a robust metric for "Deep Intersaliency," capturing semantic similarities that shallow features miss.

3. Shallow Intersaliency and Fusion

The deep mining results are integrated with traditional "shallow" features (like low-level color and texture matching) and the transferred intrasaliency priors. A weighted linear combination of these cues produces the final, refined cosaliency map.

Experimental Results

The model was evaluated on the Image Pair and iCoseg datasets. On the challenging iCoseg dataset, the method achieved an Average Precision (AP) of 0.878 and an Area Under the Curve (AUC) of 0.969, surpassing contemporary state-of-the-art models.

Qualitative comparison of cosaliency detection results on a baseball image group, showing the progression from input images to intermediate maps and the final output.

Qualitatively, the framework demonstrates superior robustness in suppressing cluttered backgrounds—such as grass or spectators in sports photos—that might appear frequently but are not the primary objects of interest.

Conclusion

By combining transfer learning for intrasaliency and unsupervised deep mining for intersaliency, this framework provides a powerful tool for identifying common ground in image sets. While the per-group training of SDAEs introduces some computational overhead, the ability to capture deep semantic features marks a significant advancement over previous shallow methods. Future work may focus on more sophisticated integration techniques and extending the framework to video sequences.

Summary updated

Abstract

As an interesting and emerging topic, cosaliency detection aims at simultaneously extracting common salient objects in multiple related images. It differs from the conventional saliency detection paradigm in which saliency detection for each image is determined one by one independently without taking advantage of the homogeneity in the data pool of multiple related images. In this paper, we propose a novel cosaliency detection approach using deep learning models. Two new concepts, called intrasaliency prior transfer and deep intersaliency mining, are introduced and explored in the proposed work. For the intrasaliency prior transfer, we build a stacked denoising autoencoder (SDAE) to learn the saliency prior knowledge from auxiliary annotated data sets and then transfer the learned knowledge to estimate the intrasaliency for each image in cosaliency data sets. For the deep intersaliency mining, we formulate it by using the deep reconstruction residual obtained in the highest hidden layer of a self-trained SDAE. The obtained deep intersaliency can extract more intrinsic and general hidden patterns to discover the homogeneity of cosalient objects in terms of some higher level concepts. Finally, the cosaliency maps are generated by weighted integration of the proposed intrasaliency prior, deep intersaliency, and traditional shallow intersaliency. Comprehensive experiments over diverse publicly available benchmark data sets demonstrate consistent performance gains of the proposed method over the state-of-the-art cosaliency detection methods.
BibTeX
@article{zhang2016cosaliency,
  author = {Dingwen Zhang and Junwei Han and Jungong Han and Ling Shao},
  title = {{Cosaliency Detection Based on Intrasaliency Prior Transfer and Deep Intersaliency Mining}},
  journal = {IEEE Trans. Neural Networks Learn. Syst.},
  year = {2016},
  doi = {10.1109/tnnls.2015.2495161},
}
Figure