Exploration of MPSO-Two-Stage Classification Optimization Model for Scene Images with Low Quality and Complex Semantics

Kexin Liu, Rong Wang, Xiaoou Song, Xiaobing Deng, Qingchao ZhuSensors 2024 · DOI

TL;DR

This paper introduces a novel two-stage classification model, MPSO-Two-Stage, designed to tackle the challenge of classifying low-quality and semantically complex scene images. By incorporating a pre-reconstruction stage and optimizing parameters with a hybrid PSO algorithm, the model achieves improved accuracy on real-world low-resolution images, particularly those affected by environmental conditions.

What Problem Does This Paper Address?

Current state-of-the-art scene classification models often perform best on high-definition, clear images. However, in practical applications like disaster response or public security, images are frequently low-quality due to factors such as low resolution, poor lighting, weather conditions, or motion blur. These low-quality images, especially those with complex semantic content (e.g., cluttered backgrounds, multiple objects), pose a significant challenge for existing classification methods, leading to reduced accuracy and potentially critical misclassifications. This paper aims to bridge this gap by developing a robust model that can effectively classify these challenging, real-world scene images.

Methods

The core of this work is the MPSO-Two-Stage Classification Optimization Model. This model operates in two stages:

  1. Level-1 Pre-reconstruction Stage: This stage utilizes a Short-Range Convolutional Neural Network (SRCNN) to enhance the resolution and quality of the input low-quality images. This is crucial for improving the clarity of semantic features before classification.
  2. Level-2 Classification Stage: This stage employs a Deep Convolutional Neural Network (DCNN) combined with an Extreme Learning Machine (ELM) for the actual scene classification.

The novelty lies in the Mix-optimization Particle Swarm Optimization (MPSO) algorithm, which is used to iteratively optimize the hyperparameters of both the SRCNN and DCNN-ELM modules. MPSO is an improved PSO variant that dynamically adjusts learning factors and inertia weights to accelerate convergence and improve search capabilities. This two-stage approach, coupled with MPSO optimization, aims to synergistically improve both image quality and classification accuracy.

Datasets

The authors utilized a combination of existing and self-constructed datasets:

  • Existing Datasets:
    • 15-Scene Dataset: A benchmark dataset with 15 scene categories.
  • SUN397: A large-scale dataset with 397 scene categories.
  • MIT Indoor: A dataset focusing on indoor scenes with 67 categories.
  • Self-Constructed Datasets:
    • Low-quality news scene frames: Seven categories of low-quality images derived from news footage, specifically designed to test the necessity of the pre-reconstruction stage.
  • IC Image Sets: Infrared images affected by illumination changes.
  • WD Image Sets: Images captured by cell phones affected by weather disturbances (rain, snow, haze).

These self-constructed datasets are crucial for evaluating the model's performance on real-world low-quality scenarios.

Key Insights and Ideas

  • Pre-reconstruction is essential for low-quality images: The paper strongly demonstrates that a dedicated pre-reconstruction stage significantly boosts classification accuracy, especially for images with lower PSNR values or those affected by environmental factors.
  • Outdoor scenes benefit more from reconstruction: The improvement in classification accuracy due to the pre-reconstruction stage is more pronounced for outdoor scenes, likely because they are more susceptible to degradation from weather and lighting.
  • MPSO effectively optimizes complex models: The hybrid MPSO algorithm proves effective in fine-tuning the hyperparameters of the two-stage architecture, leading to better overall performance than standard optimization methods.
  • Semantic complexity matters: While the model performs well, it's noted that scenes with very small target information and excessive interfering information can still pose challenges.

Key Results

  • Overall Accuracy Improvement: The proposed MPSO-Two-Stage model achieved a 1.54% higher accuracy than the best existing method (ResNet-ELM) on the 15-scene dataset.
  • Performance on Different Datasets: It outperformed other methods on the 15-scene and MIT-Indoor datasets, showing competitive results on the SUN dataset.
  • Necessity of Pre-reconstruction: Experiments on self-built news scene frames showed that the proposed reconstruction method yielded higher improvement rates for outdoor scenes compared to other preprocessing methods.
  • Weather and Lighting Robustness: An adaptive test on self-built outdoor scene sets affected by lighting and weather indicated suitability for such conditions, with an average accuracy improvement of 1.42%.
  • Specific Category Improvements: For poor-quality scenarios, improvements were particularly notable in "Building" (up to 4.53%) and "Fire" categories.

Limitations and Future Work

  • Reliance on Semantic Elements: The model's performance can be limited if the effective target information in an image is too small relative to interfering information, leading to semantic confusion.
  • Future Directions:
    • Develop methods to remove a large amount of interfering information in the pre-reconstruction module for more efficient recognition.
  • Introduce practical dynamic video target detection and consider consecutive keyframes for comprehensive judgment.
  • Optimize the two-stage classification by integrating architectures like SAM for scenes with distinct edge delineation and achieving automated segmentation.

Summary updated

Abstract

Currently, complex scene classification strategies are limited to high-definition image scene sets, and low-quality scene sets are overlooked. Although a few studies have focused on artificially noisy images or specific image sets, none have involved actual low-resolution scene images. Therefore, designing classification models around practicality is of paramount importance. To solve the above problems, this paper proposes a two-stage classification optimization algorithm model based on MPSO, thus achieving high-precision classification of low-quality scene images. Firstly, to verify the rationality of the proposed model, three groups of internationally recognized scene datasets were used to conduct comparative experiments with the proposed model and 21 existing methods. It was found that the proposed model performs better, especially in the 15-scene dataset, with 1.54% higher accuracy than the best existing method ResNet-ELM. Secondly, to prove the necessity of the pre-reconstruction stage of the proposed model, the same classification architecture was used to conduct comparative experiments between the proposed reconstruction method and six existing preprocessing methods on the seven self-built low-quality news scene frames. The results show that the proposed model has a higher improvement rate for outdoor scenes. Finally, to test the application potential of the proposed model in outdoor environments, an adaptive test experiment was conducted on the two self-built scene sets affected by lighting and weather. The results indicate that the proposed model is suitable for weather-affected scene classification, with an average accuracy improvement of 1.42%.
BibTeX
@inproceedings{liu2024exploration,
  author = {Kexin Liu and Rong Wang and Xiaoou Song and Xiaobing Deng and Qingchao Zhu},
  title = {{Exploration of MPSO-Two-Stage Classification Optimization Model for Scene Images with Low Quality and Complex Semantics}},
  booktitle = {Sensors},
  year = {2024},
  doi = {10.3390/s24123983},
}
Figure