Image Quality Assessment: Exploring Quality Awareness via Memory-driven Distortion Patterns Matching

Xuting Lan, Mingliang Zhou, Xuekai Wei, Jielu Yan, Yueting Huang, Huayan Pu, Jun Luo, Weijia Jia

Abstract

Existing full-reference image quality assessment (FR-IQA) methods achieve high-precision evaluation by analysing feature differences between reference and distorted images. However, their performance is constrained by the quality of the reference image, which limits real-world applications where ideal reference sources are unavailable. Notably, the human visual system has the ability to accumulate visual memory, allowing image quality assessment on the basis of long-term memory storage. Inspired by this biological memory mechanism, we propose a memory-driven quality-aware framework (MQAF), which establishes a memory bank for storing distortion patterns and dynamically switches between dual-mode quality assessment strategies to reduce reliance on high-quality reference images. When reference images are available, MQAF obtains reference-guided quality scores by adaptively weighting reference information and comparing the distorted image with stored distortion patterns in the memory bank. When the reference image is absent, the framework relies on distortion patterns in the memory bank to infer image quality, enabling no-reference quality assessment (NR-IQA). The experimental results show that our method outperforms state-of-the-art approaches across multiple datasets while adapting to both no-reference and full-reference tasks.

Introduction

In the field of Image Quality Assessment (IQA), methodologies are traditionally bifurcated into Full-Reference (FR) and No-Reference (NR) approaches. While FR-IQA provides high accuracy by comparing distorted images to pristine originals, it suffers from a "pristine-image problem." In practical scenarios—such as streaming over unstable networks or digital restoration—a high-quality reference is often unavailable or degraded. Conversely, the human visual system (HVS) can assess quality without an explicit reference by leveraging a "mental library" of visual experiences.

This paper introduces the Memory-driven Quality-Aware Framework (MQAF), which bridges this gap by incorporating a trainable memory bank of prototypical distortion patterns. This architecture allows the model to mimic human cognitive processes, providing robust assessments in both FR and NR scenarios.

Methodology

Framework Architecture

The MQAF architecture is comprised of three primary stages: feature encoding, deep domain representation, and quality-aware assessment.

The overall structure of MQAF, illustrating the data flow from input to the dual-mode quality assessment.
  1. Feature Encoder: High-level semantic features are extracted using a pre-trained backbone, such as ResNet50 or VGG16.
  2. Memory Bank: This module contains 256 trainable "memory units," each representing a 2048-dimensional vector of a prototypical distortion pattern (e.g., noise, blur, or compression artifacts). To ensure diversity, a decorrelation loss is applied to keep these units distinct.
  3. Dual-Mode Assessment:
    • Reference Mode (MQAF-R): Utilizes an Adaptive Weight Network (AWN) to dynamically balance information from the reference image and the memory bank.
  4. No-Reference Mode (MQAF-NR): Operates independently of a reference image by matching distorted features solely against the memory bank's stored patterns.

Quality-Aware Insights

A key innovation in MQAF is the recognition that feature magnitude is as critical as feature direction. While standard cosine similarity measures the angle between vectors, MQAF employs a combined distance metric that accounts for both angle and norm-based similarity, capturing subtle quality variations that angular metrics alone might miss. Furthermore, the AWN acts as a safety net; if a provided reference image is of low quality, the system automatically shifts its confidence toward the internal memory bank.

Experiments and Results

Experimental Setup

The framework was evaluated on five benchmark datasets: LIVE, CSIQ, TID2013, KADID-10K, and PIPAL. These datasets cover a wide spectrum of distortions, ranging from traditional noise to complex GAN-based artifacts. Performance was measured using Pearson Linear Correlation Coefficient (PLCC) and Spearman Rank Correlation Coefficient (SRCC).

Performance Analysis

MQAF demonstrated state-of-the-art (SOTA) performance across both FR and NR tasks. As shown in the performance table below, MQAF-R (VGG16) achieved top-tier results on LIVE and KADID-10K, often outperforming established metrics like LPIPS and DISTS.

Quantitative performance comparison of MQAF against SOTA FR and NR methods across major benchmarks.

Notably, MQAF-NR (No-Reference mode) achieved results comparable to many dedicated FR algorithms. For instance, on the KADID-10K dataset, MQAF-NR (ResNet50) reached a PLCC of 0.962, significantly higher than many traditional NR-IQA methods. The model also exhibited high computational efficiency, processing the entire PIPAL dataset in approximately 2.57 minutes using a ResNet50 backbone.

Conclusion

The Memory-driven Quality-Aware Framework (MQAF) successfully integrates a biological-inspired memory mechanism into image quality assessment. By utilizing an explicit memory bank of distortion patterns, the model reduces the industry's dependency on pristine reference images while maintaining high accuracy. Future work will explore the application of this unified framework to video quality assessment, where temporal memory units could further enhance performance in dynamic environments.

Summary updated

Abstract

Existing full-reference image quality assessment (FR-IQA) methods achieve high-precision evaluation by analysing feature differences between reference and distorted images. However, their performance is constrained by the quality of the reference image, which limits real-world applications where ideal reference sources are unavailable. Notably, the human visual system has the ability to accumulate visual memory, allowing image quality assessment on the basis of long-term memory storage. Inspired by this biological memory mechanism, we propose a memory-driven quality-aware framework (MQAF), which establishes a memory bank for storing distortion patterns and dynamically switches between dual-mode quality assessment strategies to reduce reliance on high-quality reference images. When reference images are available, MQAF obtains reference-guided quality scores by adaptively weighting reference information and comparing the distorted image with stored distortion patterns in the memory bank. When the reference image is absent, the framework relies on distortion patterns in the memory bank to infer image quality, enabling no-reference quality assessment (NR-IQA). The experimental results show that our method outperforms state-of-the-art approaches across multiple datasets while adapting to both no-reference and full-reference tasks.
BibTeX
@misc{lan2026image,
  author = {Xuting Lan and Mingliang Zhou and Xuekai Wei and Jielu Yan and Yueting Huang and Huayan Pu and Jun Luo and Weijia Jia},
  title = {{Image Quality Assessment: Exploring Quality Awareness via Memory-driven Distortion Patterns Matching}},
  year = {2026},
  eprint = {2602.18000},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2602.18000},
}
Figure