Causal Perception Inspired Representation Learning for Trustworthy Image Quality Assessment

Lei Wang, Desen YuanarXiv 2024 · DOI · arXiv

Introduction

Deep learning has significantly advanced Image Quality Assessment (IQA), enabling automated systems to predict human perception of image quality with high accuracy. However, these models often lack robustness, making them susceptible to adversarial attacks that can lead to unreliable predictions in real-world applications. This paper introduces CPRL (Causal Perception inspired Representation Learning), a framework designed to enhance the trustworthiness of IQA models by leveraging causal inference.

The Vulnerability of IQA Models

Standard IQA models are highly vulnerable to adversarial perturbations—mathematically calculated noise that is invisible to the human eye but catastrophic for AI evaluators.

Adversarial attack on IQA: adding imperceptible noise significantly degrades the predicted quality score.

As illustrated above, adding a minute amount of noise can cause a state-of-the-art model like DBCNN to drop its quality score from 0.8763 to 0.5010. This vulnerability stems from the model's reliance on "non-robust" features—statistical shortcuts or textures that correlate with quality in training data but do not represent the actual cause of perceived quality. Traditional defenses, such as passive adversarial training or denoising, often fail because they can inadvertently alter the quality features they are meant to protect.

The CPRL Framework

To address this, the authors propose a causal framework that distinguishes between two types of image features:

  1. Causal Perception Representation (CPR): Features that are the true cause of subjective quality (e.g., sharpness, structural integrity). These are invariant to minor perturbations.
  2. Non-Causal Perception Representation (N-CPR): "Confounders" that create spurious correlations (e.g., local textures) which adversarial attacks exploit.
The CPRL framework architecture, featuring the causal representation learning module and optimization strategy.

Key Methodological Innovations

  • CPRL Module: The framework replaces standard ReLU activation layers with a module utilizing a soft ranking-based channel-wise activation function. This acts as a gatekeeper, amplifying "causally sufficient" features while suppressing non-causal ones.
  • PNS Risk Optimization: The model is optimized using the Probability of Necessity and Sufficiency (PNS). This involves a minimax game where the model maximizes the representativeness of causal features while minimizing the influence of confounders.
  • Backdoor Adjustment: By treating adversarial attacks as "backdoor" interventions in a causal graph, the model can "intervene" during training to break spurious correlations between the image and its quality label.

Experimental Results

The CPRL method was evaluated on four major IQA benchmarks: LIVE and VCL (synthetic distortions), and LIVEC and KONIQ (authentic distortions).

Robustness Under Attack

CPRL significantly outperformed state-of-the-art (SOTA) defense methods under various attack scenarios:

  • FGSM Attack: On the VCL dataset, while standard models saw their SROCC (Spearman Rank Order Correlation Coefficient) drop to near zero, CPRL maintained a high 0.9041.
  • PGD Attack: On the LIVE dataset, CPRL achieved an SROCC of 0.8864, far exceeding the next best defense method (LWTA at 0.5887).

Clean Performance

Importantly, CPRL does not sacrifice accuracy on "clean" (un-attacked) images. It maintains performance comparable to or better than standard SOTA models, such as an SROCC of 0.9292 on the VCL dataset.

Conclusion

By mimicking the "all or nothing" principle of human perception—where minor interference is ignored until a threshold is reached—CPRL creates a more stable and trustworthy IQA model. While the framework introduces some computational overhead during training, its ability to maintain high accuracy under adversarial pressure marks a significant step forward in robust image quality assessment. Future work will explore more complex causal interventions and extend this framework to other trustworthy AI domains.

Summary updated

BibTeX
@misc{wang2024causal,
  author = {Lei Wang and Desen Yuan},
  title = {{Causal Perception Inspired Representation Learning for Trustworthy Image Quality Assessment}},
  year = {2024},
  eprint = {2404.19567},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2404.19567},
}
Figure