Image Quality Assessment: Investigating Causal Perceptual Effects with Abductive Counterfactual Inference

Wenhao Shen, Mingliang Zhou, Yu Chen, Xuekai Wei, Jun Luo, Huayan Pu, Weijia Jia

TL;DR

This paper introduces a novel approach to Full-Reference Image Quality Assessment (FR-IQA) by framing it as a counterfactual inference problem. By investigating the causal relationships between deep network features and perceptual distortions, their method achieves more robust and interpretable quality predictions that generalize better across diverse datasets.

What Problem Does This Paper Address?

Traditional Image Quality Assessment (IQA) methods, especially those relying on deep learning, often struggle to capture the complex, non-linear ways human perception is affected by image distortions. While these methods excel at identifying correlations between image features and perceived quality, they can fall short in understanding the underlying causal mechanisms. This limitation hinders their ability to generalize to new datasets or distortion types and makes their quality scores less interpretable. The authors aim to bridge this gap by explicitly modeling the causal relationships between image features and human perception, moving beyond mere statistical similarity.

Methods

The core of this work lies in applying abductive counterfactual inference to FR-IQA. Here's a breakdown of their approach:

  • Causal Feature Analysis: Instead of just comparing feature representations of reference and distorted images, they investigate how these features causally influence perceived quality. They use a Structural Causal Model (SCM) to represent these relationships.
  • Counterfactual Interventions: To validate causal relationships, they employ causal interventions. This involves manipulating specific features (using the do() operator) to see how the predicted quality changes. If a feature's change significantly alters the predicted quality, it's considered causally relevant.
  • Confounder Dictionary: An unknown exogenous variable U is introduced to account for uncertainties and identify representations that are truly causally related to quality, separating them from incidental noise or spurious correlations.
  • Perceptual Cost Metric: They define a causal transport cost to quantify perceptual differences, emphasizing the minimal perceptual cost associated with distortions.
  • Backbone Independence: A key advantage is that the analysis of perceptual causal correlations is independent of the specific deep learning backbone used for feature extraction (e.g., VGG, ResNet, EfficientNet). This allows for broad applicability.

Datasets

The authors evaluated their method on six widely used IQA databases:

  • LIVE [27]
  • CSIQ [16]
  • TID2008 [21]
  • TID2013 [22]
  • KADID [19]
  • PIPAL [10]

They note that for some methods, training datasets were used, while their proposed method is training-free. They also mention using a linear normalization process to standardize MOS scores across these diverse datasets.

Key Insights and Ideas

  • Causality Over Correlation: The fundamental insight is that understanding the causal impact of distortions on perception is more crucial for robust IQA than simply measuring statistical correlations between features.
  • Generalizability Through Causality: By focusing on causal relationships, the model becomes less sensitive to dataset-specific biases and spurious correlations, leading to better generalization.
  • Interpretability Boost: The causal framework provides a clearer understanding of why a certain quality score is assigned, making the model more interpretable.
  • Distortion-Specific Causal Effects: The method can reveal how different distortions affect specific perceptual channels in varying ways, offering deeper insights into human visual perception.

Key Results

  • Superior Performance: The proposed method consistently achieves superior overall performance and aligns closely with actual subjective scores across multiple benchmark datasets, outperforming many state-of-the-art methods.
  • Robustness: The method demonstrates robustness and effectiveness, providing competitive quality predictions.
  • Generalizability: The training-free nature and backbone independence contribute to its strong generalizability across diverse datasets and distortion types.
  • Ablation Studies: Experiments show that incorporating causal features significantly improves predictive performance compared to models relying solely on non-causal representations.

Limitations and Future Work

  • Real-World Distortions: The authors plan to further explore the causal effects in real-world distortions, suggesting that their current work might be more focused on synthetic distortions.
  • Other Vision Enhancement Tasks: They suggest investigating the application of their causal model in other vision enhancement tasks, indicating potential for broader impact beyond IQA.
  • Interpretability of SCM: While the SCM is used, the paper focuses on its application rather than a deep dive into the interpretability of the learned causal graph itself.

The paper's contribution lies in shifting the paradigm of IQA from correlation-based to causality-based reasoning, offering a more principled and effective way to assess image quality.

Summary updated

Abstract

Existing full-reference image quality assessment (FR-IQA) methods often fail to capture the complex causal mechanisms that underlie human perceptual responses to image distortions, limiting their ability to generalize across diverse scenarios. In this paper, we propose an FR-IQA method based on abductive counterfactual inference to investigate the causal relationships between deep network features and perceptual distortions. First, we explore the causal effects of deep features on perception and integrate causal reasoning with feature comparison, constructing a model that effectively handles complex distortion types across different IQA scenarios. Second, the analysis of the perceptual causal correlations of our proposed method is independent of the backbone architecture and thus can be applied to a variety of deep networks. Through abductive counterfactual experiments, we validate the proposed causal relationships, confirming the model's superior perceptual relevance and interpretability of quality scores. The experimental results demonstrate the robustness and effectiveness of the method, providing competitive quality predictions across multiple benchmarks. The source code is available at https://anonymous.4open.science/r/DeepCausalQuality-25BC.
BibTeX
@inproceedings{shen2025image,
  author = {Wenhao Shen and Mingliang Zhou and Yu Chen and Xuekai Wei and Jun Luo and Huayan Pu and Weijia Jia},
  title = {{Image Quality Assessment: Investigating Causal Perceptual Effects with Abductive Counterfactual Inference}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2025},
  doi = {10.1109/cvpr52734.2025.01676},
}
Figure