Nested Network With Two-Stream Pyramid for Salient Object Detection in Optical Remote Sensing Images

Chongyi Li, Runmin Cong, Junhui Hou, Sanyi Zhang, Yue Qian, Sam KwongIEEE TGRS 2019 · DOI · OpenReview

Abstract

Salient object detection (SOD) in optical remote sensing images (RSIs) is a challenging task due to extreme variations in object scale, diverse imaging orientations, and highly cluttered backgrounds. Traditional SOD methods designed for natural scenes often fail when applied to the complex overhead perspectives of satellite and aerial imagery. This paper introduces LV-Net, an end-to-end deep learning architecture specifically engineered for the RSI domain. LV-Net utilizes a two-stream pyramid (L-shaped module) to extract multi-scale complementary information and a nested encoder-decoder structure (V-shaped module) to suppress background noise. To facilitate research in this area, we also introduce the ORSSD dataset, the first publicly available benchmark for RSI saliency detection. Experimental results demonstrate that LV-Net significantly outperforms existing state-of-the-art methods.

Introduction

Salient Object Detection (SOD) aims to identify the most visually distinctive regions in an image, mimicking the human visual attention mechanism. While SOD has matured significantly for "natural scene" images (standard eye-level photography), its application to Optical Remote Sensing Images (RSIs) remains difficult.

RSIs present three primary hurdles that standard models struggle to overcome:

  1. Extreme Scale Variation: Salient objects can range from tiny vehicles to massive industrial complexes within the same dataset.
  2. Orientation Diversity: Unlike consumer photos, which have a predictable "upright" bias, objects in RSIs appear at any orientation.
  3. Cluttered Backgrounds: Complex terrains, shadows, and atmospheric interference (like clouds) create significant noise that can be easily mistaken for salient objects.

To address these challenges, we propose LV-Net, a purely data-driven framework designed to handle the unique geometric and environmental characteristics of remote sensing data.

The LV-Net Architecture

The proposed LV-Net is named after the geometric shapes formed by its two core components: the L-shaped two-stream pyramid module and the V-shaped nested encoder-decoder module.

L-Shaped Module: Two-Stream Pyramid

To effectively perceive objects across diverse scales, the L-shaped module processes images through two parallel paths:

  • Input Pyramid: This stream uses down-sampled versions of the original image to preserve fine-grained spatial details. This is critical for detecting small objects that might otherwise be lost in the deep layers of a standard neural network.
  • Feature Pyramid: Extracted via Multi-Scale Convolution Units, this stream captures high-level semantic context, allowing the model to understand the "what" and "where" of larger salient regions.

V-Shaped Module: Nested Connections

Standard encoder-decoder networks (like U-Net) typically use simple skip connections to pass information. However, in RSIs, these connections can inadvertently pass background noise (such as shadows or waves) to the final output. LV-Net employs nested connections that act as a series of filters. These connections gradually integrate detail features from the encoder with semantic features from the decoder, effectively "sieving out" clutter and sharpening the boundaries of the detected objects.

The ORSSD Dataset

A significant contribution of this work is the introduction of the Optical Remote Sensing Saliency Detection (ORSSD) dataset. Prior to this, the field lacked a dedicated, high-quality benchmark. The ORSSD dataset includes:

  • 800 Images: Featuring diverse spatial resolutions and various saliency types (e.g., ships, bridges, aircraft, and stadiums).
  • Pixel-wise Ground Truth: High-quality manual annotations for precise evaluation.
  • Real-world Complexity: The dataset includes challenging scenarios, such as images containing no salient objects at all, forcing models to be more robust against false positives.

Experimental Results

LV-Net was evaluated against 14 state-of-the-art methods, including both unsupervised models and deep learning models designed for natural scenes.

Quantitative Performance

LV-Net achieved superior results across all major metrics on the ORSSD benchmark:

  • F-measure: 0.8414 (the highest among compared models).
  • Mean Absolute Error (MAE): 0.0207 (the lowest among compared models).

Efficiency and Qualitative Analysis

Despite its sophisticated nested architecture, LV-Net remains computationally efficient with a model size of 207 MB and an inference speed of 0.74 seconds per image. Qualitatively, the model demonstrates a unique ability to isolate multiple objects of varying sizes in a single frame while ignoring complex background textures like forest canopies or urban grids.

Conclusion

LV-Net represents a significant step forward for saliency detection in the remote sensing domain. By combining a two-stream pyramid for scale awareness with nested connections for noise suppression, the model overcomes the inherent difficulties of RSI analysis. Future work will focus on improving edge sharpness through dedicated context modules and expanding the ORSSD dataset to include even more diverse geographic scenes.

Summary updated

Abstract

Arising from the various object types and scales, diverse imaging orientations, and cluttered backgrounds in optical remote sensing image (RSI), it is difficult to directly extend the success of salient object detection for nature scene image to the optical RSI. In this paper, we propose an end-to-end deep network called LV-Net based on the shape of network architecture, which detects salient objects from optical RSIs in a purely data-driven fashion. The proposed LV-Net consists of two key modules, i.e., a two-stream pyramid module (L-shaped module) and an encoder-decoder module with nested connections (V-shaped module). Specifically, the L-shaped module extracts a set of complementary information hierarchically by using a two-stream pyramid structure, which is beneficial to perceiving the diverse scales and local details of salient objects. The V-shaped module gradually integrates encoder detail features with decoder semantic features through nested connections, which aims at suppressing the cluttered backgrounds and highlighting the salient objects. In addition, we construct the first publicly available optical RSI dataset for salient object detection, including 800 images with varying spatial resolutions, diverse saliency types, and pixel-wise ground truth. Experiments on this benchmark dataset demonstrate that the proposed method outperforms the state-of-the-art salient object detection methods both qualitatively and quantitatively.
BibTeX
@article{li2019nested,
  author = {Chongyi Li and Runmin Cong and Junhui Hou and Sanyi Zhang and Yue Qian and Sam Kwong},
  title = {{Nested Network With Two-Stream Pyramid for Salient Object Detection in Optical Remote Sensing Images}},
  journal = {IEEE Trans. Geosci. Remote. Sens.},
  year = {2019},
  doi = {10.1109/tgrs.2019.2925070},
}
Figure