Salient Object Detection Based on Optimization of Feature Computation by Neutrosophic Set Theory

Sensen Song, Yue Li, Zhenhong Jia, Fei ShiSensors 2023 · DOI

TL;DR

This paper introduces a novel salient object detection method that optimizes feature computation using neutrosophic set (NS) theory. By carefully selecting features and leveraging prior object knowledge, the method aims to improve saliency map accuracy and reduce uncertainty, outperforming existing state-of-the-art approaches on several benchmark datasets.

What Problem Does This Paper Address?

The core challenge addressed is the difficulty in accurately detecting salient objects in images. Existing methods often struggle with either using too many or too few image features, leading to unsatisfactory saliency map details and degraded detection results. This can manifest as either over-segmentation (too many features) or under-segmentation (missing crucial information). Furthermore, saliency maps often contain fuzzy or uncertain information, making precise object segmentation difficult. The paper aims to overcome these limitations by proposing a more robust feature selection and optimization process.

Methods

The proposed method, SMNS, consists of two main parts:

  1. Salient Object Detection:
    • Prior Object Knowledge Construction: This involves building foreground and background models using pixel-wise and super-pixel cues. This step aims to capture essential object information.
  2. Feature Extraction and Selection: The paper emphasizes selecting features based on their uniqueness rather than quantity. Several low-level features are explored, including color features (L*a*b* color space), local image entropy, guided image filtering, and gradients. A specific combination, denoted as "2*", is found to be optimal.
  3. Feature Computation and Separation: A low-rank matrix decomposition technique is employed to separate image features into low-rank (background) and sparse (salient object) components. This is a modification of existing methods, using a single-layer structure for efficiency.
  4. Feature Fusion: The sparse features from low-rank decomposition are fused with the prior object knowledge to refine the salient object information.
  5. Saliency Optimization:
    • Neutrosophic Set (NS) Theory: The paper introduces a novel mathematical definition for NS theory, redefining its three subsets: True (T), Indeterminacy (I), and False (F). This theory is used to handle the inherent fuzziness and uncertainty in saliency maps.
  6. Optimized Saliency Map Generation: The T, I, and F components are used to process the saliency detection results, aiming to highlight object features and separate them from the background. A fusion step combines these optimized components to produce the final, clean saliency map.

Datasets

The proposed algorithm was evaluated on five popular benchmark datasets:

  • MSRA10K: Contains 10,000 images with simple scenes and obvious objects.
  • ECSSD: Contains 1000 images with simple scenes and obvious objects.
  • PASCAL-S: Composed of 850 images with relatively complex backgrounds and multiple salient objects.
  • DUT-OMRON: Contains 5168 images with complex backgrounds and multiple salient objects.
  • SOCK: Derived from SOC, containing 1800 images of various sizes with complex backgrounds and multiple salient objects, excluding images with no salient object.

Key Insights and Ideas

  • Feature Quantity vs. Quality: The research strongly suggests that the number of features is less important than their suitability and rational combination for the specific algorithm. Too many features can lead to redundancy and noise, while too few can result in incomplete information.
  • The Power of Prior Knowledge: Incorporating prior object knowledge (foreground and background models) significantly aids in separating salient objects from their surroundings.
  • Neutrosophic Sets for Uncertainty: NS theory proves to be a powerful tool for handling the inherent fuzziness and uncertainty in saliency maps, leading to cleaner and more accurate results. The novel mathematical definition of NS theory is a key contribution.
  • Efficiency through Simplification: Modifying the low-rank matrix decomposition by removing a complex tree structure into a single-layer approach improves computational efficiency without sacrificing performance.

Key Results

The SMNS algorithm demonstrated competitive and superior performance across the evaluated datasets.

  • Quantitative Metrics: The paper reports strong results across various metrics, including MAE, S-measure, and AUC. For instance, on the MSRA10K dataset, the SMNS algorithm achieved an MAE of 0.097, an S-measure of 0.830, and an AUC of 0.851.
  • Comparison to State-of-the-Art: The SMNS method consistently outperformed or was highly competitive with several state-of-the-art traditional theory-based methods on the benchmark datasets.
  • Feature Combination Impact: Experiments showed that feature combination "2*" yielded the best results, highlighting the importance of feature selection.
  • NS Theory Optimization: Results indicated a significant improvement in saliency detection accuracy and quality when the NS theory optimization was applied, outperforming results without this step.
  • Efficiency: The simplified low-rank matrix decomposition achieved a faster processing time (1.205s per image on ECSSD) compared to methods with a tree structure (1.524s).

Limitations and Future Work

  • Failure Cases in Complex Scenes: The paper acknowledges that the algorithm can still fail in complex scenes where objects blend with their surroundings, making accurate detection challenging.
  • Parameter Tuning: While parameters are set and fixed for experiments, the paper notes that different parameter settings can impact results, and further optimization and exploration of parameter relationships are warranted.
  • Brain-Inspired Techniques: The authors suggest that future work could explore developing brain-inspired techniques for salient object detection, drawing inspiration from the biological brain's effectiveness in this area and the psychological underpinnings of NS theory.
  • Extending NS Theory: The methodological versatility of NS theory suggests potential applications beyond saliency detection, such as in robotics, video analysis, and content creation.

Summary updated

Abstract

In recent saliency detection research, too many or too few image features are used in the algorithm, and the processing of saliency map details is not satisfactory, resulting in significant degradation of the salient object detection result. To overcome the above deficiencies and achieve better object detection results, we propose a salient object detection method based on feature optimization by neutrosophic set (NS) theory in this paper. First, prior object knowledge is built using foreground and background models, which include pixel-wise and super-pixel cues. Simultaneously, the feature maps are selected and extracted for feature computation, allowing the object and background features of the image to be separated as much as possible. Second, the salient object is obtained by fusing the features decomposed by the low-rank matrix recovery model with the object prior knowledge. Finally, for salient object detection, we present a novel mathematical description of neutrosophic set theory. To reduce the uncertainty of the obtained saliency map and then obtain good saliency detection results, the new NS theory is proposed. Extensive experiments on five public datasets demonstrate that the results are competitive and superior to previous state-of-the-art methods.
BibTeX
@inproceedings{song2023salient,
  author = {Sensen Song and Yue Li and Zhenhong Jia and Fei Shi},
  title = {{Salient Object Detection Based on Optimization of Feature Computation by Neutrosophic Set Theory}},
  booktitle = {Sensors},
  year = {2023},
  doi = {10.3390/s23208348},
}
Figure