TL;DR
This paper introduces a novel framework that uses salient regions of an image to guide blind image sharpness assessment (BISA). They found that focusing BISA models on these salient regions can achieve performance comparable to, and sometimes better than, using the entire image, especially when compared to using just the image center.
What Problem Does This Paper Address?
Assessing image sharpness without a reference image (Blind Image Sharpness Assessment or BISA) is crucial for many applications, from automated image quality control to user experience enhancement. Image blur is a common distortion that significantly degrades visual quality. Traditional BISA methods often analyze the entire image, which can be computationally expensive and might dilute the impact of important details. The human visual system, however, naturally focuses on "salient" regions – the most visually interesting or informative parts of a scene. This paper investigates whether leveraging this human-like attention mechanism by focusing BISA models on salient regions can improve sharpness assessment accuracy and efficiency.
Methods
The core of this work is a hybrid framework that combines Salient Region Detection (SRD) methods with existing Blind Image Sharpness Assessment (BISA) models.
- Salient Region Detection (SRD): They employed three established SRD methods:
- DPLSG: Diffusion process on a 2-layer sparse graph.
- SORBD: Saliency optimization from robust background detection.
-
SRIS: Saliency region detection and image segmentation. These methods generate "saliency maps" that highlight the most visually important areas of an image.
-
Input Reorganization: The saliency maps were then used to create new input regions for the BISA models:
- Whole Image (WI): The standard input.
- Center Region (CR): A baseline input, based on the prior that salient objects are often centered.
- Salient Region (SR): The region directly identified by the saliency map.
- Major Salient Region (mSR): A slightly expanded salient region.
- BISA Evaluation: Ten traditional (non-deep learning) BISA models were used to assess image sharpness on these different input regions. The performance was evaluated using Pearson linear correlation coefficient (PLCC) for accuracy and Spearman rank-order correlation coefficient (SRCC) for monotonicity.
Datasets
The study utilized three publicly available datasets, all containing images degraded by Gaussian blurring:
- CSIQ: Contains 30 pristine images.
- LIVE: Contains 29 pristine images.
- TID13: Contains 25 pristine images.
These datasets were chosen for their diversity in image content, including faces, nature scenes, and man-made objects. The blurring was synthesized intentionally at various levels.
Key Insights and Ideas
- Salient Regions are Informative for Sharpness: The human visual system's tendency to focus on salient regions isn't just for scene understanding; it's also highly relevant for assessing image sharpness.
- Center Region is Insufficient: Relying solely on the center of an image (a common prior in SRD) is not enough for effective sharpness assessment, as it often misses crucial edge structures present in salient areas.
- SORBD is a Strong Performer: The SORBD method for saliency detection consistently produced salient regions that led to better BISA performance across various BISA models, regardless of the dataset.
- Efficiency Gains: Using salient regions as input can significantly reduce the computational burden for BISA models, as they process smaller image portions.
- SRD Methods Matter: Different SRD methods yield varying results. While DPLSG sometimes led to performance drops, SORBD and SRIS generally maintained or improved BISA scores.
Key Results
- Comparable to Whole Image: In most cases, using salient regions (SR and mSR) as input for BISA models achieved performance (PLCC/SRCC) comparable to using the entire image (WI).
- Superior to Center Region: Salient region inputs consistently outperformed the center region (CR) input across all datasets and most BISA models.
- Specific Model Improvements: For example, on the LIVE dataset, the MEW model showed a significant increase in PLCC (up to 0.159) and SRCC (up to 0.130) when using salient regions compared to the CR input.
- Computational Efficiency: SORBD was identified as a good balance between saliency detection accuracy and computational speed, requiring less than 1 second per image.
Limitations and Future Work
- Dataset Specificity: The study focused on Gaussian blurring. Future work should explore other types of image distortions.
- SRD Method Limitations: Current SRD methods often rely on image biases and contrast, which might not be ideal for blurred images. The impact of blur on saliency map detection needs further investigation.
- No Deep Learning: The study did not evaluate deep-learning-based SRD or BISA models, partly due to challenges with consistent output sizes from SRD methods for deep learning inputs. Future work could explore how to integrate these advanced models.
- SRD/BISA Integration: The paper suggests that how SRD methods are incorporated into BISA models needs further exploration to maximize performance gains.
- Saliency vs. Fixation: The authors note that salient regions might differ from human fixation regions, suggesting a need for careful design when incorporating SRD into image quality assessment.
Summary updated