Abstract
Salient object detection (SOD) aims to mimic the human visual system (HVS) and cognition mechanisms to identify and segment salient objects. However, due to the complexity of these mechanisms, current methods are not perfect. Accuracy and robustness need to be further improved, particularly in complex scenes with multiple objects and background clutter. To address this issue, we propose a novel approach called Multiple Enhancement Network (MENet) that adopts the boundary sensibility, content integrity, iterative refinement, and frequency decomposition mechanisms of HVS. A multi-level hybrid loss is firstly designed to guide the network to learn pixel-level, region-level, and object-level features. A flexible multiscale feature enhancement module (ME-Module) is then designed to gradually aggregate and refine global or detailed features by changing the size order of the input feature sequence. An iterative training strategy is used to enhance boundary features and adaptive features in the dual-branch decoder of MENet. Comprehensive evaluations on six challenging benchmark datasets show that MENet achieves state-of-the-art results.
Introduction
Salient Object Detection (SOD) is a fundamental task in computer vision that involves identifying and segmenting the most visually prominent objects in an image. While deep learning has significantly advanced the field, existing models often struggle in complex environments characterized by geometric complexity, background clutter, and the inherent tension between maintaining object integrity and capturing fine details.
Humans naturally balance boundary sensitivity with global content integrity. Current SOD methods frequently fail to replicate this balance, often producing "leaky" segmentations in noisy backgrounds or losing thin structures. To bridge this gap, we introduce the Multiple Enhancement Network (MENet), which incorporates mechanisms inspired by the Human Visual System (HVS) to improve both accuracy and robustness.
Methods
MENet utilizes a U-Net-like architecture enhanced by several novel components designed to mimic human cognitive processes.
Dual-Branch Decoder and ME-Module
Instead of a standard single-path decoder, MENet employs a two-stream approach. One branch focuses on gradients (high-frequency boundary information), while the other processes adaptive features (low-frequency internal object information). By decoupling these streams, the model prevents noisy boundary data from corrupting the global consistency of the object's body.
Central to this architecture is the Multiscale Feature Enhancement Module (ME-Module). This module utilizes Atrous Spatial Pyramid Pooling (ASPP) and attention mechanisms. Its primary innovation is its flexibility: by altering the size order of the input feature sequence, the module can be toggled to prioritize either global feature aggregation or local detail refinement.
Iterative Refinement Strategy
The network employs a four-round iterative training strategy, mimicking a "look closer, then step back" heuristic. Odd-numbered rounds focus on extracting global features to establish object wholeness, while even-numbered rounds refine these features with local details to sharpen boundaries.
Multi-level Hybrid Loss
To ensure the model learns at various granularities, we implement a hybrid loss function:
- Pixel-level: Standard Binary Cross Entropy (BCE) for point-wise accuracy.
- Region-level: Employs SSIM and IoU across four sub-regions to enforce structural consistency within local areas.
- Object-level: A specialized loss inspired by the S-measure that evaluates the contrast and distribution of the entire foreground object against the ground truth.
Datasets and Evaluation
The model was pre-trained on ImageNet and fine-tuned on DUTS-TR. We evaluated MENet across six benchmark datasets: OMRON, DUTS-TE, HKU-IS, PASCAL-S, ECSSD, and SOD. These datasets represent a wide range of challenges, including multiple scattered objects, complex backgrounds, and semantically complex structures.
Key Results
MENet demonstrates superior performance compared to 16 recent state-of-the-art methods, including ICON and EGNet.
Quantitative and Qualitative Performance
On the OMRON dataset, MENet achieved a state-of-the-art Mean Absolute Error (MAE) of 0.0450. Qualitatively, the model excels at suppressing background noise and maintaining sharp, closed boundaries, even when the salient object's appearance is highly similar to its surroundings.
Efficiency
Despite its iterative nature, MENet is highly efficient, achieving a processing speed of 45 FPS on an NVIDIA A100. This makes it suitable for real-time applications where both high accuracy and speed are required.
Conclusion
MENet advances salient object detection by integrating HVS-inspired mechanisms into a deep learning framework. Through its dual-branch decoder, flexible ME-Module, and multi-level hybrid loss, it effectively balances global object integrity with local detail. While low-contrast and blurred scenes remain a challenge for future research, MENet sets a new benchmark for robustness in complex SOD tasks.
Summary updated