Subjective and Objective Quality Evaluation for Underwater Image Enhancement and Restoration

Wenxia Li, Chi Lin, Ting Luo, Hong Li, Haiyong Xu, Lihong WangSymmetry 2022 · DOI

TL;DR

This paper introduces a new benchmark dataset and an objective quality evaluation metric (UIQEI) for underwater images that have undergone enhancement or restoration. The UIQEI leverages novel features derived from dark and bright channel priors, color, fog density, and global contrast to accurately assess image quality, outperforming existing methods.

What Problem Does This Paper Address?

Underwater images are notoriously difficult to capture due to the complex optical properties of water, leading to issues like low visibility, color distortion, and blur. While many methods exist to enhance or restore these images, they often result in over- or under-enhancement, making it hard to objectively compare their effectiveness. The authors identify a critical need for robust underwater image quality evaluation (UIQE) methods that can accurately assess these processed images, especially for guiding the development of better enhancement and restoration techniques.

Methods

The core of this work lies in two main contributions:

  1. Subjective Evaluation Dataset (UIQE): The authors created a dataset of 405 enhanced underwater images. These were generated by applying 9 different enhancement/restoration methods (including traditional, deep learning, and restoration techniques) to 45 real underwater images. This dataset was then used for subjective quality evaluation by human observers.
  2. Objective Quality Evaluation Index (UIQEI): A novel no-reference (NR) image quality assessment (IQA) metric is proposed. The UIQEI is built upon the idea of "relative symmetry" between two key underwater image priors:
  3. Underwater Dark Channel Prior (UDCP): This prior leverages the observation that in underwater images, at least one color channel (typically red) decays rapidly, leading to dark regions. The UDCP focuses on these darker areas.
  4. Underwater Bright Channel Prior (UBCP): This is presented as a symmetrical counterpart to the UDCP, focusing on bright regions or areas of supersaturation.

    The UIQEI extracts features from these priors, along with color saturation, fog density, and global contrast. These features are then fused using a regression model (specifically, Support Vector Regression - SVR) to predict the image quality score.

Datasets

  • U45 Dataset: A collection of 45 real underwater images used as the source for generating the enhanced images. These images represent various underwater scenes, lighting conditions, and depths.
  • UIQE Dataset: The primary dataset introduced in this paper. It comprises 405 enhanced underwater images, created by applying 9 different enhancement/restoration methods to the 45 source images. This dataset was used for both subjective evaluation and as a benchmark for the proposed objective metric.

Key Insights and Ideas

  • The "Symmetry" of Dark and Bright Channels: The paper highlights the complementary nature of dark and bright channel priors in underwater imaging. While dark channels capture information in shadowed or color-attenuated regions, bright channels reveal details in overexposed or highly saturated areas. Their combined analysis offers a more comprehensive view of image degradation.
  • Feature Fusion is Crucial: The UIQEI's success stems from fusing multiple feature types (UDCP/UBCP-derived GM/LOG statistics, color, fog, contrast). This suggests that no single feature is sufficient to capture the complex quality degradation in underwater images.
  • Subjective Evaluation is Still King (for Benchmarking): The creation of a subjectively evaluated dataset was essential for training and validating the objective metric. This underscores the importance of human perception in defining "quality."
  • Existing Methods Struggle with Balance: The subjective evaluation revealed that even advanced enhancement methods often fail to strike a balance between removing color cast and preserving fine details, leading to over- or under-enhancement.

Key Results

  • UIQEI Performance: The proposed UIQEI achieved a high SROCC of 0.8568 and PLCC of 0.8705 on the UIQE dataset, with a low RMSE of 0.3600. This significantly outperforms several existing no-reference IQA metrics (BRISQUE, NIQE, UCIQE, UIQM, CCF, ILNIQE) in predicting underwater image quality.
  • Feature Importance: Ablation studies indicated that features derived from GM and LOG statistics (especially under UDCP) contribute most significantly to the UIQEI's performance, followed by fog density.
  • Robustness: The UIQEI demonstrated good performance across different underwater scenes (blue, green, haze) and showed stability even with a smaller training set percentage (as low as 40%).

Limitations and Future Work

  • Dataset Size: While 405 images is a good start, the authors suggest building larger datasets, potentially including more diverse ocean types, for even more robust evaluation.
  • Generalization: The paper focuses on specific types of underwater image degradation. Future work could explore extending the UIQEI to handle a wider range of underwater conditions and artifacts.
  • Computational Cost: While not explicitly detailed as a limitation, the extraction and fusion of multiple features might have computational implications for real-time applications.
  • Qualitative vs. Quantitative Balance: The authors emphasize that a combination of qualitative and quantitative evaluation is crucial for a comprehensive understanding of underwater image enhancement and restoration methods.

Summary updated

Abstract

Since underwater imaging is affected by the complex water environment, it often leads to severe distortion of the underwater image. To improve the quality of underwater images, underwater image enhancement and restoration methods have been proposed. However, many underwater image enhancement and restoration methods produce over-enhancement or under-enhancement, which affects their application. To better design underwater image enhancement and restoration methods, it is necessary to research the underwater image quality evaluation (UIQE) for underwater image enhancement and restoration methods. Therefore, a subjective evaluation dataset for an underwater image enhancement and restoration method is constructed, and on this basis, an objective quality evaluation method of underwater images, based on the relative symmetry of underwater dark channel prior (UDCP) and the underwater bright channel prior (UBCP) is proposed. Specifically, considering underwater image enhancement in different scenarios, a UIQE dataset is constructed, which contains 405 underwater images, generated from 45 different underwater real images, using 9 representative underwater image enhancement methods. Then, a subjective quality evaluation of the UIQE database is studied. To quantitatively measure the quality of the enhanced and restored underwater images with different characteristics, an objective UIQE index (UIQEI) is used, by extracting and fusing four groups of features, including: (1) the joint statistics of normalized gradient magnitude (GM) and Laplacian of Gaussian (LOG) features, based on the underwater dark channel map; (2) the joint statistics of normalized gradient magnitude (GM) and Laplacian of Gaussian (LOG) features, based on the underwater bright channel map; (3) the saturation and colorfulness features; (4) the fog density feature; (5) the global contrast feature; these features capture key aspects of underwater images. Finally, the experimental results are analyzed, qualitatively and quantitatively, to illustrate the effectiveness of the proposed UIQEI method.
BibTeX
@inproceedings{li2022subjective,
  author = {Wenxia Li and Chi Lin and Ting Luo and Hong Li and Haiyong Xu and Lihong Wang},
  title = {{Subjective and Objective Quality Evaluation for Underwater Image Enhancement and Restoration}},
  booktitle = {Symmetry},
  year = {2022},
  doi = {10.3390/sym14030558},
}
Figure