Abstract
Tone mapping operators (TMOs) and multi-exposure fusion methods are essential for displaying high dynamic range (HDR) content on standard dynamic range (SDR) devices. However, these processes often introduce visual distortions. We propose a novel no-reference method for tone-mapped image quality assessment (TMIQA) that utilizes multi-scale and multi-layer features extracted from a pre-trained deep convolutional neural network (DCNN). By aggregating features across different scales and network depths, our model captures both local textures and global semantic information. Experimental results on the ESPL-LIVE HDR database demonstrate that our approach significantly outperforms existing state-of-the-art no-reference methods.
Introduction
High Dynamic Range (HDR) imaging captures a broader range of luminance than traditional formats, but most consumer displays are limited to Standard Dynamic Range (SDR). Bridging this gap requires Tone Mapping Operators (TMOs) or multi-exposure fusion techniques. While these methods make HDR content viewable, they can cause loss of detail, unnatural color shifts, or halo artifacts.
Evaluating the quality of these tone-mapped images is critical. While full-reference metrics exist, they require the original HDR source, which is often unavailable in end-user applications. This necessitates No-Reference Image Quality Assessment (NR-IQA) models that can predict perceived quality based solely on the tone-mapped image.
Proposed Methodology
The complexity of distortions in tone-mapped images requires a multi-faceted approach. Our framework leverages a pre-trained ResNet-50 model to extract rich feature representations.
1. Multi-Scale Representation
DCNNs typically have a fixed receptive field. To capture global structural distortions alongside fine-grained local artifacts, we input both the original image and a down-sampled version into the network. This dual-scale approach ensures the model "sees" the image at different levels of granularity.
2. Multi-Layer Feature Extraction
Instead of relying only on the final layer, we extract features from multiple stages of the ResNet-50 architecture:
- Low-level features: Capture fundamental textures and edges.
- Mid-to-High-level features: Capture complex shapes and semantic scene information.
3. Feature Aggregation and Prediction
The extracted feature maps are aggregated using mean and standard deviation pooling to form a comprehensive feature vector. Given the high dimensionality (4,608) relative to the dataset size, we employ Partial Least Squares Regression (PLSR) to map these features to a final quality score. PLSR effectively handles high-dimensional data and prevents overfitting.
Experimental Setup and Datasets
We evaluated our model on the ESPL-LIVE HDR Database, the largest public dataset for TMIQA. It contains 1,811 images generated from 11 different processing pipelines.
The database provides Mean Opinion Scores (MOS) for each image, representing the collective judgment of over 5,000 human participants. This serves as the ground truth for training and testing our quality predictor.
Results and Discussion
The performance was measured using Spearman’s Rank Correlation Coefficient (SROCC) to determine how well the model's rankings align with human perception.
- Superior Performance: Our method achieved an SROCC of 0.823, outperforming specialized NR-IQA methods like HIGRADE (0.730) and BIBQA (0.702).
- Domain Specificity: General-purpose NR-IQA models like BRISQUE (0.418) performed poorly, highlighting that tone-mapping distortions are distinct from common noise or blur.
- Ablation Study: Removing either the multi-scale or multi-layer components resulted in a performance drop, confirming that both are vital for accurate quality prediction.
Conclusion
By combining multi-scale inputs with multi-layer DCNN features, our proposed method provides a robust solution for assessing the quality of tone-mapped images without a reference. This approach effectively captures the diverse range of distortions introduced by TMOs, matching human visual preference more closely than previous techniques.
Summary updated