Traditional image quality metrics like Peak Signal-to-Noise Ratio (PSNR) operate on a simple, flawed assumption: every pixel in an image is equally important. To a computer, a compression artifact in a clear blue sky is mathematically identical to one across a person's eye. To a human, however, the latter is a dealbreaker, while the former might go unnoticed.
Recent research has shifted toward Attention-Aware Image Quality Assessment (IQA). This direction treats image quality not as a global average of errors, but as a weighted sum where the weights are determined by the Human Visual System (HVS). By modeling where humans look (saliency) and how sensitive they are to specific regions (masking), these models align automated scores with subjective human experience.
The Failure of Uniform Pooling
The core problem in IQA is "pooling"—the process of collapsing a map of local errors into a single quality score. Fei Zhou and colleagues argue in Image Quality Assessment Based on Multi-Order Visual Comparison that simple averages fail because humans have non-uniform sensitivity. We are biologically wired to prioritize regions with significant structural information or severe degradation.
To fix this, researchers like Jongyoo Kim and Sanghoon Lee developed DeepQA, which learns a "visual sensitivity map." This map acts as a gatekeeper: it highlights distortions in smooth, clear areas where they are "revealed" and suppresses them in complex textures where they are "masked." This mimics the eye's natural tendency to ignore noise in a field of grass while recoiling from a glitch on a smooth forehead.
Artifacts as Visual Attractors
If we know where a human is looking, we can predict which errors will bother them. However, Zhengzhong Tu points out a subtle complication in Predicting Eye Fixations Under Distortion Using Bayesian Observers: distortions themselves change where we look.
Heavy JPEG "blockiness" or ringing artifacts act as "competitive visual attractors." They hijack our attention, pulling our eyes away from the intended subject of the photo. By treating local distortion levels as a feature alongside natural saliency (like contrast or faces), models can predict how artifacts distract the viewer, leading to a more accurate assessment of "annoyance."
Transformers as Saliency Engines
The rise of the Transformer architecture has provided a powerful tool for this research. Unlike CNNs, which focus on local patches, Transformers use self-attention to model global relationships. Manri Cheon and his team demonstrated in Perceptual Image Quality Assessment With Transformers that these attention maps naturally function as saliency maps.
When a Transformer evaluates an image, its internal attention weights highlight the regions of degradation it finds most salient. Jinsong Shi expanded on this with TempQT, using a dedicated "perceptual quality token." This token aggregates information from across the image, weighting patches based on their perceived impact. This allows the model to distinguish between the physical presence of noise and its subjective impact on the viewer.
Foveated Assessment in Immersive Media
The need for attention-aware metrics is most acute in 360-degree video and Virtual Reality. In these environments, a user only sees a small "viewport" at any given time. Kanglong Fan and Kede Ma address this in Learned Scanpaths Aid Blind Panoramic Video Quality Assessment by training a probabilistic generator to mimic human viewing patterns.
By sampling only the viewports a human is likely to inhabit, the model avoids wasting "error budget" on parts of the sphere the user will never see. This concept is pushed to its logical conclusion by Aaron B. Wagner and Johannes Ballé in their work on Wasserstein Distortion. They propose a framework that unifies fidelity and realism: images should be reconstructed with high precision in salient (foveal) regions but can rely on statistical synthesis in the periphery.
Domain-Specific Priors
Finally, certain subjects—most notably faces—demand specialized attention models. Shaolin Su and colleagues found that general-purpose metrics are often "face-blind," attending to irrelevant background details. Their model, GFIQA-20k, uses generative priors to understand what a "perfect" face should look like. This allows the system to prioritize the features humans care about most—eyes, noses, and mouths—ensuring that quality scores reflect the social and biological importance of the subject matter.