Sliced Maximal Information Coefficient: A Training-Free Approach for Image Quality Assessment Enhancement

Kang Xiao, Xu Wang, Yulin He, Baoliang Chen, Xuelin Shen

Introduction

Full-Reference Image Quality Assessment (FR-IQA) is a critical component in digital imaging, aimed at automatically quantifying the degradation of a distorted image relative to a pristine reference. Traditional metrics like Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index (SSIM) have long served as industry standards. However, these metrics frequently fail to align with human visual perception, particularly when dealing with complex distortions or GAN-generated artifacts.

The primary limitation of existing FR-IQA methods lies in their pooling strategy. Most algorithms calculate local error maps and then apply a simple uniform average to produce a final score. This approach ignores the nuances of the Human Visual System (HVS), which naturally prioritizes certain image regions—such as faces or foreground objects—while disregarding others. While attention-based models have been proposed to address this, they typically require extensive training on labeled datasets and often struggle to generalize across diverse distortion types.

The SMIC Framework

To bridge the gap between algorithmic scoring and human perception without the need for costly training, researchers have introduced the Sliced Maximal Information Coefficient (SMIC). This "plug-and-play" module estimates human attention by measuring the statistical dependency between reference and distorted images in a deep feature space.

Methodology

The core innovation of SMIC is its use of statistical dependency as a proxy for visual attention. The framework operates on the intuition that areas of high dependency between the reference and distorted images are "predictable" and thus less likely to capture human interest. Conversely, areas where the dependency is broken indicate "noticeable" changes that draw visual attention.

The SMIC framework pipeline: Deep features from a pre-trained VGG16 are processed through the SMIC module to generate an attention map, which rectifies the distortion map before final pooling.

The process involves three key steps:

  1. Feature Extraction: Both the reference ($I^r$) and distorted ($I^d$) images are passed through a pre-trained VGG16 network to extract high-dimensional deep features.
  2. Sliced MIC Calculation: Because the standard Maximal Information Coefficient (MIC) is designed for 1D data, SMIC employs "random projections." High-dimensional features are sliced into multiple 1D representations using random Gaussian weights. The MIC is calculated for each slice and then averaged to estimate the overall dependency.
  3. Attention Mapping: The resulting dependency map is transformed into an attention map ($M^a$) using a simple heuristic: $M^a = 1 - SMIC$. This map is then used to weight the local distortion maps ($M^d$) of base metrics like PSNR or LPIPS, creating a "rectified" map ($M^r$) that emphasizes perceptually significant errors.

Experimental Results

The SMIC enhancement was rigorously tested across six major IQA benchmarks, including LIVE, CSIQ, TID2013, KADID-10k, and specialized datasets like PIPAL (GAN-based distortions) and QADS (super-resolution artifacts).

Quantitative Performance

The results demonstrate that SMIC consistently improves the performance of every base metric it is paired with. Notably, the enhancement is most pronounced on datasets featuring complex distortions.

Quantitative comparison of various IQA methods with and without SMIC enhancement across six benchmarks. Blue text indicates the percentage improvement in correlation metrics.

Key highlights from the performance data include:

  • Significant Correlation Boosts: On the TID2013 dataset, the Spearman Rank Correlation Coefficient (SRCC) for LPIPS increased from 0.670 to 0.867—a nearly 30% improvement.
  • Revitalizing Classic Metrics: By integrating SMIC, the basic PSNR metric was able to outperform several modern, complex state-of-the-art models on multiple datasets.
  • Robustness to GAN Artifacts: On the PIPAL dataset, which is notoriously difficult for traditional metrics due to the presence of realistic but unfaithful GAN textures, SMIC-enhanced metrics showed substantial gains in both SRCC and PLCC.

Discussion and Conclusion

The SMIC approach offers a compelling alternative to data-hungry attention models. Its training-free nature ensures that it does not overfit to specific datasets or distortion types, allowing it to generalize effectively across a wide range of imaging scenarios. By modeling attention as a lack of statistical dependency in deep feature space, SMIC provides a mathematically grounded yet computationally feasible way to align automated quality scores with human judgment.

While the current implementation relies on VGG16 features and a linear attention mapping, the framework is inherently flexible. Future research may explore the use of more advanced backbones, such as Vision Transformers, or more sophisticated non-linear mappings to further refine the estimation of human visual attention. Ultimately, SMIC represents a significant step toward more "perceptually aware" image processing pipelines that can be deployed without the overhead of specialized training.

Summary updated

Abstract

Full-reference image quality assessment (FR-IQA) models generally operate by measuring the visual differences between a degraded image and its reference. However, existing FR-IQA models including both the classical ones (eg, PSNR and SSIM) and deep-learning based measures (eg, LPIPS and DISTS) still exhibit limitations in capturing the full perception characteristics of the human visual system (HVS). In this paper, instead of designing a new FR-IQA measure, we aim to explore a generalized human visual attention estimation strategy to mimic the process of human quality rating and enhance existing IQA models. In particular, we model human attention generation by measuring the statistical dependency between the degraded image and the reference image. The dependency is captured in a training-free manner by our proposed sliced maximal information coefficient and exhibits surprising generalization in different IQA measures. Experimental results verify the performance of existing IQA models can be consistently improved when our attention module is incorporated. The source code is available at https://github.com/KANGX99/SMIC.
BibTeX
@inproceedings{xiao2024sliced,
  author = {Kang Xiao and Xu Wang and Yulin He and Baoliang Chen and Xuelin Shen},
  title = {{Sliced Maximal Information Coefficient: A Training-Free Approach for Image Quality Assessment Enhancement}},
  booktitle = {ICME},
  year = {2024},
  doi = {10.1109/icme57554.2024.10687355},
}
Figure