TL;DR
This paper introduces a novel two-stage classification model, MPSO-Two-Stage, designed to tackle the challenge of classifying low-quality and semantically complex scene images. By incorporating a pre-reconstruction stage and optimizing parameters with a hybrid PSO algorithm, the model achieves improved accuracy on real-world low-resolution images, particularly those affected by environmental conditions.
What Problem Does This Paper Address?
Current state-of-the-art scene classification models often perform best on high-definition, clear images. However, in practical applications like disaster response or public security, images are frequently low-quality due to factors such as low resolution, poor lighting, weather conditions, or motion blur. These low-quality images, especially those with complex semantic content (e.g., cluttered backgrounds, multiple objects), pose a significant challenge for existing classification methods, leading to reduced accuracy and potentially critical misclassifications. This paper aims to bridge this gap by developing a robust model that can effectively classify these challenging, real-world scene images.
Methods
The core of this work is the MPSO-Two-Stage Classification Optimization Model. This model operates in two stages:
- Level-1 Pre-reconstruction Stage: This stage utilizes a Short-Range Convolutional Neural Network (SRCNN) to enhance the resolution and quality of the input low-quality images. This is crucial for improving the clarity of semantic features before classification.
- Level-2 Classification Stage: This stage employs a Deep Convolutional Neural Network (DCNN) combined with an Extreme Learning Machine (ELM) for the actual scene classification.
The novelty lies in the Mix-optimization Particle Swarm Optimization (MPSO) algorithm, which is used to iteratively optimize the hyperparameters of both the SRCNN and DCNN-ELM modules. MPSO is an improved PSO variant that dynamically adjusts learning factors and inertia weights to accelerate convergence and improve search capabilities. This two-stage approach, coupled with MPSO optimization, aims to synergistically improve both image quality and classification accuracy.
Datasets
The authors utilized a combination of existing and self-constructed datasets:
- Existing Datasets:
- 15-Scene Dataset: A benchmark dataset with 15 scene categories.
- SUN397: A large-scale dataset with 397 scene categories.
- MIT Indoor: A dataset focusing on indoor scenes with 67 categories.
- Self-Constructed Datasets:
- Low-quality news scene frames: Seven categories of low-quality images derived from news footage, specifically designed to test the necessity of the pre-reconstruction stage.
- IC Image Sets: Infrared images affected by illumination changes.
- WD Image Sets: Images captured by cell phones affected by weather disturbances (rain, snow, haze).
These self-constructed datasets are crucial for evaluating the model's performance on real-world low-quality scenarios.
Key Insights and Ideas
- Pre-reconstruction is essential for low-quality images: The paper strongly demonstrates that a dedicated pre-reconstruction stage significantly boosts classification accuracy, especially for images with lower PSNR values or those affected by environmental factors.
- Outdoor scenes benefit more from reconstruction: The improvement in classification accuracy due to the pre-reconstruction stage is more pronounced for outdoor scenes, likely because they are more susceptible to degradation from weather and lighting.
- MPSO effectively optimizes complex models: The hybrid MPSO algorithm proves effective in fine-tuning the hyperparameters of the two-stage architecture, leading to better overall performance than standard optimization methods.
- Semantic complexity matters: While the model performs well, it's noted that scenes with very small target information and excessive interfering information can still pose challenges.
Key Results
- Overall Accuracy Improvement: The proposed MPSO-Two-Stage model achieved a 1.54% higher accuracy than the best existing method (ResNet-ELM) on the 15-scene dataset.
- Performance on Different Datasets: It outperformed other methods on the 15-scene and MIT-Indoor datasets, showing competitive results on the SUN dataset.
- Necessity of Pre-reconstruction: Experiments on self-built news scene frames showed that the proposed reconstruction method yielded higher improvement rates for outdoor scenes compared to other preprocessing methods.
- Weather and Lighting Robustness: An adaptive test on self-built outdoor scene sets affected by lighting and weather indicated suitability for such conditions, with an average accuracy improvement of 1.42%.
- Specific Category Improvements: For poor-quality scenarios, improvements were particularly notable in "Building" (up to 4.53%) and "Fire" categories.
Limitations and Future Work
- Reliance on Semantic Elements: The model's performance can be limited if the effective target information in an image is too small relative to interfering information, leading to semantic confusion.
- Future Directions:
- Develop methods to remove a large amount of interfering information in the pre-reconstruction module for more efficient recognition.
- Introduce practical dynamic video target detection and consider consecutive keyframes for comprehensive judgment.
- Optimize the two-stage classification by integrating architectures like SAM for scenes with distinct edge delineation and achieving automated segmentation.
Summary updated