SuperVAE: Superpixelwise Variational Autoencoder for Salient Object Detection

Bo Li, Zhengxing Sun, Yuqi GuoAAAI 2019 · OpenReview

Abstract

Salient Object Detection (SOD) has advanced rapidly with deep neural networks, yet challenges remain. Pixel-wise convolutional neural networks (CNNs) often produce blurry boundaries due to pooling operations, while region-based methods frequently lack spatial consistency. We propose SuperVAE, a framework using a superpixelwise variational autoencoder to model image backgrounds. By identifying salient objects through reconstruction residuals and employing a perceptual loss for spatial context, SuperVAE generates high-quality saliency maps without mask-level annotations. Experiments across five benchmark datasets demonstrate that SuperVAE achieves superior performance compared to existing unsupervised methods and remains competitive with state-of-the-art supervised models.

Introduction

Identifying the most visually prominent objects in an image—Salient Object Detection—is a fundamental task in computer vision. While deep learning has significantly improved SOD performance, most high-performing models rely on extensive supervised training with pixel-perfect manual annotations.

Existing deep learning approaches typically suffer from two main issues:

  1. Blurry Boundaries: Pixel-wise CNNs lose fine-grained spatial details during downsampling (pooling and strided convolutions).
  2. Lack of Spatial Consistency: Region-based methods process image segments independently, often failing to maintain a coherent label across a single object.

SuperVAE addresses these limitations by reframing saliency detection as an anomaly detection problem, specifically modeling the background to identify the "odd one out."

Methodology

The SuperVAE framework operates on the principle that image boundaries typically represent the background. The process involves several key steps:

Superpixel Segmentation and Background Modeling

The input image is first partitioned into superpixels using the SLIC algorithm, which naturally preserves object boundaries. Superpixels located at the image edges are selected as background samples. A Variational Autoencoder (VAE) is then trained on these samples to learn the generative distribution of the specific image's background.

Saliency via Reconstruction Residual

Once the VAE is trained, it attempts to reconstruct every superpixel in the image. Because the model has only learned the characteristics of the background, it fails to accurately reconstruct salient foreground objects. The resulting reconstruction error, or "residual," serves as the saliency score for each region.

Perceptual Loss and Multi-scale Fusion

To ensure spatial consistency, the authors incorporate a perceptual loss based on a pre-trained VGG19 network. This allows the model to capture high-level semantic and spatial context rather than just raw pixel values. Finally, the process is executed at multiple superpixel scales and fused to handle objects of varying sizes.

Results and Discussion

SuperVAE shows a significant performance leap over previous unsupervised techniques. On the ECSSD dataset, it improved the F-measure by 17.5% and reduced the Mean Absolute Error (MAE) by 34.39% compared to the leading unsupervised algorithms.

Qualitative comparison of SuperVAE (OURS) against Ground Truth (GT) and state-of-the-art supervised methods (MDF, RFCN) on two sample images.

As shown in the figure above, SuperVAE (column e) produces saliency maps that are remarkably close to the ground truth (column b). Notably, it maintains sharper boundaries and better internal consistency than supervised methods like MDF (column c) and RFCN (column d), despite not using any ground-truth masks during its training phase.

Conclusion

SuperVAE demonstrates that modeling the background through a generative approach is a powerful strategy for unsupervised salient object detection. By combining the boundary-preserving nature of superpixels with the semantic awareness of perceptual loss, the framework achieves results competitive with supervised models. While it currently relies on the boundary prior—which may fail if an object touches all edges—it represents a major step toward high-quality, label-free saliency detection.

Summary updated

Abstract

Image saliency detection has recently witnessed rapid progress due to deep neural networks. However, there still exist many important problems in the existing deep learning based methods. Pixel-wise convolutional neural network (CNN) methods suffer from blurry boundaries due to the convolutional and pooling operations. While region-based deep learning methods lack spatial consistency since they deal with each region independently. In this paper, we propose a novel salient object detection framework using a superpixelwise variational autoencoder (SuperVAE) network. We first use VAE to model the image background and then separate salient objects from the background through the reconstruction residuals. To better capture semantic and spatial contexts information, we also propose a perceptual loss to take advantage from deep pre-trained CNNs to train our SuperVAE network. Without the supervision of mask-level annotated data, our method generates high quality saliency results which can better preserve object boundaries and maintain the spatial consistency. Extensive experiments on five wildly-used benchmark datasets show that the proposed method achieves superior or competitive performance compared to other algorithms including the very recent state-of-the-art supervised methods.
BibTeX
@inproceedings{li2019supervae,
  author = {Bo Li and Zhengxing Sun and Yuqi Guo},
  title = {{SuperVAE: Superpixelwise Variational Autoencoder for Salient Object Detection}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2019},
  url = {https://openreview.net/forum?id=Gkv4F9yJJz},
}
Figure