Less Yet Robust: Crucial Region Selection for Scene Recognition

Jianqi Zhang, Mengxuan Wang, Jingyao Wang, Lingyu Si, Changwen Zheng, Fanjiang Xu

TL;DR

Researchers have developed a "plug-and-play" adaptive selection mechanism that forces neural networks to focus on the most discriminative regions of an image while ignoring distracting noise. By using a learnable mask and sparsity-inducing regularization, the method improves scene recognition accuracy and significantly boosts robustness in challenging environments like underwater and remote sensing.

What Problem Does This Paper Address?

Scene recognition—identifying whether an image depicts a "beach," "forest," or "rocky seabed"—is a foundational task in computer vision. While modern Convolutional Neural Networks (CNNs) are powerful, they often struggle with "in-the-wild" data, such as underwater or aerial imagery, which frequently suffers from blurring, overexposure, or occlusions.

The authors point out a specific flaw in standard CNNs: they tend to extract "panoramic" features, meaning they look at the whole image. In low-quality images, this is a liability. For example, if a sea anemone is obscuring a rock, a standard ResNet might focus on the anemone (the distraction) rather than the rock (the label). The challenge is to teach models to identify and rely only on the most "crucial" and "robust" high-level semantic regions, effectively filtering out the noise that leads to misclassification.

Methods

The authors propose a novel architecture centered around a Learnable Mask Matrix ($M$).

  1. Feature Masking: The mask is applied to the high-level feature map ($E$) generated by a backbone network (like ResNet or MobileNet) using a Hadamard product ($E_s = E \odot M$). This allows the model to "weight" different regions, preserving important features (values near 1) and suppressing irrelevant ones (values near 0).
  2. Importance Regularization: To ensure the model focuses on fewer regions (the "Less" in the title), they introduce an $L1$-norm regularization term to the loss function. This encourages sparsity in the mask, pushing the network to be selective rather than looking at everything.
  3. Plug-and-Play Design: The mechanism is lightweight and can be integrated into almost any existing CNN-based classifier without requiring massive architectural overhauls.

Datasets

The study evaluates the method on two primary benchmarks:

  • UGS (Underwater Geological Scene): A new dataset introduced by the authors. It contains high-resolution photos (3648 × 2736) taken by the Fendouzhe deep-sea submersible. It focuses on two categories: sediment and rock.
  • UCM (UC Merced Land Use): A standard remote sensing dataset containing 21 categories of aerial imagery (e.g., agricultural, baseball diamond, intersection).

Key Insights and Ideas

  • "Less is More" for Robustness: By restricting the model's "field of view" to a smaller set of highly discriminative features, the model becomes naturally more resistant to noise. If the noise occurs in a region the mask has already suppressed, it cannot affect the final prediction.
  • Addressing Category Sharing: Some image regions might contain features common to multiple categories, which often causes "confusion" in standard models. The learnable mask helps the network identify and down-weight these ambiguous regions in favor of unique, category-specific markers.
  • Visual Evidence of Focus: Using Grad-CAM visualizations, the authors show that their method produces much tighter, more accurate attention maps. While a standard ResNet-18 might look at a large, vague area to identify a "baseball diamond," the masked version zooms in precisely on the diamond itself.

Key Results

The proposed method consistently outperformed baseline models across various architectures:

  • Accuracy Boost: On the UGS dataset, adding the mask to ResNet-18 improved average accuracy from 90.0% to 94.0%. On the UCM dataset, it improved from 96.4% to 97.3%.
  • Noise Resilience: The most striking results appeared in robustness tests. When Gaussian noise or salt-and-pepper noise was added to images, the masked models maintained significantly higher accuracy than their baseline counterparts. As noise levels increased, the performance gap between the masked and unmasked models widened, proving the mask's effectiveness at filtering interference.
  • Model Agnostic: The improvements were consistent across ResNet, MobileNet, and even some Vision Transformer (ViT) variants.

Limitations and Future Work

While the results are promising, there are a few areas for further exploration:

  • Dataset Scale: The newly introduced UGS dataset is relatively small (two classes). Testing the mask on more diverse underwater datasets with more categories would further validate its utility.
  • Computational Overhead: Although the mask is a simple matrix multiplication, the paper does not explicitly detail the impact on inference latency, which is critical for real-time applications like autonomous underwater vehicles (AUVs).
  • Hyperparameter Sensitivity: While the authors claim the model isn't overly sensitive to the regularization weight ($\lambda$), finding the "sweet spot" for sparsity versus accuracy might require tuning for different specific domains.

Summary updated

Abstract

Scene recognition, particularly for aerial and underwater images, often suffers from various types of degradation, such as blurring or overexposure. Previous works that focus on convolutional neural networks have been shown to be able to extract panoramic semantic features and perform well on scene recognition tasks. However, low-quality images still impede model performance due to the inappropriate use of high-level semantic features. To address these challenges, we propose an adaptive selection mechanism to identify the most important and robust regions with high-level features. Thus, the model can perform learning via these regions to avoid interference. implement a learnable mask in the neural network, which can filter high-level features by assigning weights to different regions of the feature matrix. We also introduce a regularization term to further enhance the significance of key high-level feature regions. Different from previous methods, our learnable matrix pays extra attention to regions that are important to multiple categories but may cause misclassification and sets constraints to reduce the influence of such regions. This is a plug-and-play architecture that can be easily extended to other methods. Additionally, we construct an Underwater Geological Scene Classification dataset to assess the effectiveness of our model. Extensive experimental results demonstrate the superiority and robustness of our proposed method over state-of-the-art techniques on two datasets.
BibTeX
@inproceedings{zhang2025less,
  author = {Jianqi Zhang and Mengxuan Wang and Jingyao Wang and Lingyu Si and Changwen Zheng and Fanjiang Xu},
  title = {{Less Yet Robust: Crucial Region Selection for Scene Recognition}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2025},
  doi = {10.1109/icassp49660.2025.10888005},
}
Figure