When training a neural network to recognize objects, the ground truth is clear-cut: a bounding box around a car represents a car. However, when training a model to evaluate human-centric visual properties—such as image quality, aesthetics, or "interestingness"—the ground truth becomes highly elusive.
Traditionally, deep learning models for these tasks rely on Mean Opinion Scores (MOS) or pairwise human preferences. Gathering these subjective labels requires large-scale user studies, which are notoriously expensive, slow, and prone to high inter-subject variance. This creates a severe subjective label bottleneck that restricts the scale and generalizability of supervised models.
To overcome this data scarcity, researchers have shifted toward paradigms that learn to perceive visual quality without relying on absolute human scores. By leveraging synthetic data, natural scene statistics, self-supervision, and vision-language alignment, modern architectures can mimic human perceptual judgment entirely from unannotated data.
Learning from Synthetic Distortions and Relative Rankings
If asking a human to assign an absolute quality score (e.g., "7.2 out of 10") to a degraded image is unscalable, how can we teach a model what a clean image looks like? The key intuition is that while absolute quality is subjective, relative quality degradation is deterministic.
We can programmatically apply synthetic distortions—such as Gaussian blur, white noise, or compression artifacts—to a pristine image at varying, known intensities. Because the exact level of degradation is controlled, we can automatically generate relative quality rankings without any human intervention.
[Pristine Image] ---> [Blur Level 1] ---> [Blur Level 2] ---> [Blur Level 3]
| | |
+-------- Ranking Engine -------------+
|
(Deterministic Labels)
In RankIQA, Xialei Liu et al. demonstrated that a Siamese network trained purely to rank these synthetic distortions learns powerful, "quality-aware" features. This ranking-based pre-training provides a robust initialization that can later be mapped to human scores with minimal fine-tuning.
To make this process more adaptable, Hancheng Zhu et al. proposed MetaIQA, which frames learning from synthetic distortions as a meta-learning problem. By treating different distortion types as distinct tasks, the model develops a generalized "quality prior" that can rapidly adapt to unknown, real-world distortions with only a handful of examples. Similarly, Gaofeng Meng et al. leveraged reinforcement learning in No-Reference Image Quality Assessment with Reinforcement Recursive List-Wise Ranking to optimize list-wise rankings recursively, outperforming many models trained directly on human data.
Opinion-Unaware Assessment via Natural Scene Statistics
Another approach entirely bypasses synthetic data by asking: What makes an image look "natural" to the human eye? Human visual perception evolved to process the statistical regularities of the natural world. When an image undergoes distortion, these underlying statistical patterns are disrupted.
"Opinion-unaware" methods exploit this disruption by modeling the distribution of features extracted from pristine, unannotated images. In Opinion-Unaware Blind Image Quality Assessment using Multi-Scale Deep Feature Statistics, Keyan Ding et al. combined the representational capacity of pre-trained deep networks with classical multivariate Gaussian statistical modeling. By measuring how far a test image's feature statistics deviate from the "natural" baseline, the model quantifies quality without ever seeing a human score during training.
To better capture how the human brain processes these statistics, Yixuan Li et al. introduced Deep Shape-Texture Statistics for Completely Blind Image Quality Evaluation. Their model fuses shape-biased and texture-biased deep features, mimicking the human visual system's dual reliance on structural boundaries and surface details to identify visual anomalies. On a broader scale, Sangnie Bhardwaj et al. developed the Perceptual Information Metric (PIM), an unsupervised information-theoretic metric that discovers human-like visual preferences purely by analyzing the statistical transitions across frames in natural videos.
Self-Supervised and Contrastive Quality Spaces
Self-supervised learning allows models to map out a continuous "quality space" by contrasting images against one another. The intuition here is that a robust quality metric should map images with similar distortion types and intensities close together, while pushing different distortions apart.
[Distortion A, Level 1] <--- (Minimize Distance) ---> [Distortion A, Level 2]
^
| (Maximize Distance)
v
[Distortion B, Level 3]
This principle is the foundation of CONTRIQUE, a framework introduced by Pavan C Madhusudana et al. in Image Quality Assessment Using Contrastive Learning. By training a convolutional neural network to identify distortion types and levels via a contrastive loss, the model learns a frozen feature space where a simple linear regressor can accurately predict quality.
For dynamic content, Zihao Yu et al. applied this to video in Video Quality Assessment Based on Swin TransformerV2 and Coarse to Fine Strategy. They utilized a dual-loss approach: a group contrastive loss to distinguish broad bitrate differences across videos, paired with a rank loss to capture subtle, frame-by-frame quality variations.
Exploiting Implicit Geometric Proxies
In specialized domains, researchers have discovered that network architectures can contain hidden, implicit proxies for quality that require no explicit annotation to uncover.
Consider the task of evaluating hand-drawn sketches. In Finding Badly Drawn Bunnies, LAN YANG et al. discovered a fascinating geometric property in classification networks. When a model is trained to classify sketches using a Geometry-Aware Classification Layer (GACL), the magnitude ($L_2$ norm) of the resulting feature vector naturally correlates with drawing quality.
Class Boundary -----------------------------------------
^ ^
| (Short Vector) | (Long Vector)
v v
[Crude Scribble] [Well-Drawn Sketch]
Because a well-drawn sketch retains clear, canonical geometric structures, it sits deeper within its class territory, producing a longer feature vector. A crude scribble, being ambiguous, hovers near the decision boundary, producing a shorter vector. As detailed by Kaiyue Pang et al. in Annotation-Free Human Sketch Quality Assessment, this mathematical byproduct completely eliminates the need for human-annotated "good" or "bad" sketch labels.
Vision-Language Models as Subjective Proxies
The emergence of large Vision-Language Models (VLMs) offers a completely different strategy: leveraging the collective human judgment implicitly embedded within billions of web-scale text-image pairs. Instead of predicting an abstract scalar score, these models can evaluate quality through natural language.
In Blind Image Quality Assessment via Vision-Language Correspondence: A Multitask Learning Perspective, Weixia Zhang et al. introduced LIQE, a framework that utilizes vision-language alignment to describe and rate image distortions simultaneously. Moving beyond simple ratings, Zhiyuan You et al. developed DepictQA-Wild (Descriptive Image Quality Assessment in the Wild), which instructs VLMs to output detailed linguistic descriptions of physical flaws, mimicking an expert human critique.
This text-driven paradigm also scales to abstract aesthetic evaluations. In AesPrompt: Zero-Shot Image Aesthetics Assessment With Multi-Granularity Aesthetic Prompt Learning, Xiangfei Sheng et al. used an ensemble of Large Language Models to convert raw textual commentary into structured training prompts. By teaching the model to navigate aesthetics hierarchically—from theme to emotion to critique—they achieved state-of-the-art zero-shot performance without ever exposing the model to a manual numerical aesthetic score.
Key Foundational Work
Learning without Human Scores for Blind Image Quality Assessment (2013) — contributed 1 concept elements to this direction.
Using Web Photos for Measuring Video Frame Interestingness (2009) — contributed 1 concept elements to this direction.