A lightweight multi-scale context network for salient object detection in optical remote sensing images

Yuhan Lin, Han Sun, Ningzhong Liu, Yetong Bian, Jun Cen, Huiyu Zhou

Abstract

Salient Object Detection (SOD) in Optical Remote Sensing Images (RSIs) is a critical task for identifying visually significant objects in satellite and aerial imagery. However, existing methods often struggle with dramatic scale variations, complex backgrounds, and high computational costs. This paper presents MSCNet, a lightweight deep learning architecture designed for efficient and accurate SOD. By integrating a Multi-Scale Context Extraction (MSCE) module and an Attention-based Pyramid Feature Aggregation (APFA) scheme, MSCNet achieves state-of-the-art performance with only 3.26 million parameters, making it suitable for deployment on resource-constrained platforms like UAVs and mobile devices.

Introduction

Salient Object Detection (SOD) aims to automatically locate and segment the most visually "attractive" or significant objects within an image. While SOD has matured in the context of natural scene images (e.g., standard photography), Optical Remote Sensing Images (RSIs) present a unique set of challenges that traditional algorithms fail to address effectively:

  1. Dramatic Scale Variation: Objects in RSIs can range from tiny vehicles to expansive, slender features like rivers or runways that span the entire frame.
  2. Complex Backgrounds: Top-down perspectives often include confusing shadows, varied terrain, and cluttered surroundings that act as distractors.
  3. Computational Constraints: High-performing models are typically massive, requiring significant memory and processing power, which limits their utility for real-time applications on the edge (e.g., on-satellite processing).

MSCNet was developed to bridge the gap between high-precision segmentation and low computational overhead, specifically tailored for the complexities of remote sensing data.

Methodology

MSCNet utilizes a MobileNet V2 backbone to maintain a lightweight profile. The architecture is enhanced by two primary innovations designed to handle the geometric and semantic complexities of RSIs.

Multi-Scale Context Extraction (MSCE) Module

To address the issue of varying object sizes, the MSCE module processes features through three parallel asymmetric convolution branches with kernel sizes of 3x3, 5x5, and 7x7. This multi-branch approach allows the network to capture contextual information at multiple scales simultaneously. Furthermore, a channel reweighting branch—utilizing global pooling and fully connected layers—is integrated to highlight the most informative feature channels, ensuring the model focuses on relevant object characteristics.

Attention-based Pyramid Feature Aggregation (APFA)

Standard decoder architectures often lose fine-grained details during the fusion of deep and shallow features. MSCNet introduces the APFA mechanism, which functions as an inverted pyramid to fuse features from adjacent layers step-by-step. This mechanism employs a dual-attention strategy:

  • Channel Attention: Applied to deep features to capture high-level semantic meaning (the "what").
  • Spatial Attention: Applied to shallow features to preserve precise boundaries and suppress background noise (the "where").

Experimental Evaluation

Datasets

The model was rigorously evaluated on two benchmark datasets:

  • ORSSD: A dataset comprising 800 challenging remote sensing images with pixel-level annotations.
  • EORSSD: An expanded version containing 2,000 images with more complex scenes and semantically diverse objects.

Performance and Efficiency

MSCNet demonstrates a superior balance between accuracy and efficiency. On the EORSSD dataset, it achieved a Max F-measure of 0.8806 and an E-measure of 0.9684. These results outperform 11 state-of-the-art models designed for natural scenes and remain competitive with much larger specialized RSI models.

Crucially, MSCNet achieves these results with only 3.26 million parameters. For comparison, this is approximately 1/9th the size of other specialized models such as DAFNet (29.35M parameters). This efficiency proves that architectural innovation, such as the use of asymmetric convolutions and targeted attention mechanisms, can effectively replace raw model depth and size.

Conclusion

MSCNet provides a robust and efficient solution for salient object detection in remote sensing. By leveraging multi-scale context and a sophisticated attention-based aggregation scheme, the model successfully identifies objects across varying scales while maintaining a minimal computational footprint.

While the model excels at object localization and noise suppression, future work will focus on improving boundary precision. The authors suggest that integrating dedicated edge detection modules could further refine the segmentation of object borders, which is essential for high-precision mapping and urban planning applications.

Summary updated

Abstract

Due to the more dramatic multi-scale variations and more complicated foregrounds and backgrounds in optical remote sensing images (RSIs), the salient object detection (SOD) for optical RSIs becomes a huge challenge. However, different from natural scene images (NSIs), the discussion on the optical RSI SOD task still remains scarce. In this paper, we propose a multi-scale context network, namely MSCNet, for SOD in optical RSIs. Specifically, a multi-scale context extraction module is adopted to address the scale variation of salient objects by effectively learning multi-scale contextual information. Meanwhile, in order to accurately detect complete salient objects in complex backgrounds, we design an attention-based pyramid feature aggregation mechanism for gradually aggregating and refining the salient regions from the multi-scale context extraction module. Extensive experiments on two benchmarks demonstrate that MSCNet achieves competitive performance with only 3.26M parameters. The code will be available at https://github.com/NuaaYH/MSCNet.
BibTeX
@inproceedings{lin2022lightweight,
  author = {Yuhan Lin and Han Sun and Ningzhong Liu and Yetong Bian and Jun Cen and Huiyu Zhou},
  title = {{A lightweight multi-scale context network for salient object detection in optical remote sensing images}},
  booktitle = {ICPR},
  year = {2022},
  url = {https://openreview.net/forum?id=kHWUfcyDEU},
}
Figure