Quantifying image similarity using measure of enhancement by entropy

Eric A. Silva, Karen Panetta, Sos С. AgaianProceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE 2007 · DOI

TL;DR

This paper introduces SEME, a novel and computationally efficient image similarity measure derived from an entropy-based enhancement metric. SEME demonstrates strong performance in quantifying image quality across various distortion types, outperforming traditional methods like MSE and PSNR, and offering a speed advantage over state-of-the-art SSIM.

What Problem Does This Paper Address?

Accurately assessing image quality is crucial for many applications, especially in image compression. Traditional objective measures like Mean Squared Error (MSE) and Peak Signal-to-Noise Ratio (PSNR) are fast but struggle to capture human perception of quality, particularly when comparing images with different types of distortions. While more advanced methods like Structural Similarity (SSIM) offer better accuracy and consistency across distortions, they come with a significant computational cost. This creates a need for image similarity measures that are both accurate across diverse distortions and computationally efficient, mimicking the human visual system (HVS) more effectively.

Methods

The core contribution of this paper is the development of a new similarity measure called Similarity-EME (SEME). SEME is derived from the Measurement of Enhancement by Entropy (EME), which was originally designed to assess image enhancement. The authors modified EME to measure the entropy of the error signal (the difference between a reference and a distorted image) rather than the enhancement itself. This modification, which involves using absolute values of luminances and removing the logarithmic nature of the original EME, results in a linear function that better approximates human perception. Two variants of SEME are proposed: SEME-r and SEME-m, differing in how they incorporate the mean squared error (MSE) of blocks. These measures are then evaluated against established metrics like MSE, PSNR, and SSIM using subjective human judgments.

Datasets

The primary dataset used for evaluation is the LIVE database, created by the Laboratory for Image and Video Engineering (LIVE) at The University of Texas at Austin. This database contains 233 JPEG-compressed images with varying distortion levels, each accompanied by a Difference Mean Opinion Score (DMOS) derived from human observer ratings.

Key Insights and Ideas

  • Entropy of the Error Signal: The key insight is that measuring the entropy of the difference between images, rather than just the difference itself, can provide a more nuanced understanding of perceptual similarity.
  • Bridging the Gap: SEME aims to bridge the gap between computationally simple but inaccurate metrics (MSE, PSNR) and computationally complex but accurate metrics (SSIM) by offering a balance of speed and accuracy across distortion types.
  • Distortion Type Matters: The paper strongly emphasizes that traditional metrics like MSE and PSNR fail when comparing images with different distortion types, a critical limitation for evaluating compression algorithms. SEME and SSIM, however, show better performance in this regard.
  • Logarithmic Nature of EME: Surprisingly, removing the logarithmic nature of the original EME and opting for a linear function yielded better results for similarity measurement, suggesting that a direct linear relationship with error entropy is more perceptually relevant in this context.

Key Results

  • Accuracy and Consistency: SEME-m-4 and SEME-m-3 demonstrated strong performance in terms of accuracy (correlation coefficient) and consistency (outlier ratio), with SEME-m-4 being nearly as accurate as SSIM.
  • Speed Advantage: SEME measures are significantly faster than SSIM. For instance, SEME-m-4 is approximately twice as fast as SSIM, and SEME-r-3 and SEME-r-4 are also considerably quicker.
  • Distortion Type Performance: Table 2 clearly illustrates how SEME and SSIM provide more varied and perceptually relevant rankings of image quality across different distortions (contrast stretch, mean shift, JPEG compression, blur, impulsive noise) compared to MSE and PSNR, which often assign similar scores.
  • Monotonicity: While SSIM leads in monotonicity, SEME-m-4 and SEME-m-3 also show competitive results.

Limitations and Future Work

  • Computational Complexity: While SEME is faster than SSIM, it is still more computationally intensive than MSE and PSNR. The paper notes that the speed of SEME is suitable for real-time applications.
  • Human Visual System Modeling: The authors acknowledge that a complete mathematical model of the HVS is not yet understood, and their approach is an approximation. Future work could explore further refinements to better capture HVS characteristics.
  • Parameter Tuning: The performance of SEME might be sensitive to the choice of block size and other parameters, which could be a subject for further investigation.
  • Further Validation: While the LIVE database provides a good benchmark, testing SEME on a wider variety of image datasets and distortion types could further validate its robustness.

Summary updated

Abstract

Measurement of image similarity is important for a number of image processing applications. Image similarity assessment is closely related to image quality assessment in that quality is based on the apparent differences between a degraded image and the original, unmodified image. Automated evaluation of image compression systems relies on accurate quality measurement. Current algorithms for measuring similarity include mean squared error (MSE), peak signal-to-noise ratio (PSNR), and structural similarity (SSIM). They have some limitations: such as consistent, accuracy and incur greater computational cost. In this paper, we show that a modified version of the measurement of enhancement by entropy (EME) can be used as an image similarity measure, and thus an image quality measure. Until now, EME has generally been used to measure the level of enhancement obtained using a given enhancement algorithm and enhancement parameter. The similarity-EME (SEME) is based on the EME for enhancement. We will compare SEME to existing measures over a set of images subjectively judged by humans. Computer simulations have demonstrated its promise through a set of examples, as well as comparison to both subjective ratings and state-of-the-art objective methods on a database of images compressed with JPEG.
BibTeX
@inproceedings{silva2007quantifying,
  author = {Eric A. Silva and Karen Panetta and Sos С. Agaian},
  title = {{Quantifying image similarity using measure of enhancement by entropy}},
  booktitle = {Proceedings of SPIE, the International Society for Optical Engineering/Proceedings of SPIE},
  year = {2007},
  doi = {10.1117/12.720087},
}
Figure