Scaling Perceptual Metrics Beyond Human Labels

This direction is split into other directions: subjective data scarcity (subjective label scarcity). Its last version stays readable and citable here.

Version 1 of 1166 papers169 conceptsfrozen 2026-10-04

The central challenge in Visual Quality Assessment (VQA) is the "Mean Opinion Score (MOS) bottleneck." To train a model to see like a human, researchers traditionally need thousands of images, each rated by dozens of people. As noted by Jinliang Han and Zijian Chen in their work on VITAL, this reliance on manual labeling is prohibitively expensive and prevents models from scaling to the diverse distortions found in the real world.

To bypass this, a new wave of research focuses on "opinion-unaware" or self-supervised methods. These approaches treat quality not as a label to be memorized, but as a structural property of data that can be discovered through geometry, synthetic degradation, or foundation models.

The Geometry of Quality

One of the most intuitive breakthroughs in this area is the discovery that quality is often encoded in the "strength" of a model's internal representations. In Finding Badly Drawn Bunnies, Lan Yang and Yi-Zhe Song introduced the Geometry-Aware Classification Layer (GACL). They found that the $L_2$ norm (the magnitude) of a feature vector naturally correlates with drawing quality.

The intuition is simple: a high-quality, recognizable sketch is "prototypical." It sits far from the ambiguous decision boundaries of a classifier. By simply measuring how "decisive" a model's features are, we can rank quality without a single human label. Similarly, Xin Li and Zhibo Chen demonstrated in StyleAM that the "style" of deep features—their mean and variance—shifts predictably when an image is distorted, allowing for unsupervised domain adaptation from synthetic to real-world images.

Learning from Synthetic Rank

If we don't have human scores, we can create our own "ground truth" by intentionally breaking images. While a model might not know the absolute quality of a photo, it can easily learn that a slightly blurred image is "better" than a heavily blurred one.

Rajesh Sureddi and Saman Zadtootaghaj leveraged this in TRIQA by using ordered distortion triplets. By training a model to rank these synthetic levels, it develops a "quality prior." This idea was expanded by Hancheng Zhu and Leida Li in MetaIQA, which uses meta-learning to "learn how to learn" from these synthetic distortions. The model gains a general understanding of what "degradation" looks like, which can then be fine-tuned on tiny amounts of real human data.

Quality as Information and Task-Performance

Another perspective moves away from "looking" at the image and toward "using" it. In An Unsupervised Information-Theoretic Perceptual Quality Metric, Sangnie Bhardwaj and Johannes Ballé argue that quality is tied to the statistics of natural scenes. Their Perceptual Information Metric (PIM) discovers human-like vision by looking at how much information a model can extract from natural video sequences.

Alternatively, quality can be defined by utility. Nikita Lisin and Anton Konushin improved face quality assessment by tying it to recognition confidence. If a face recognition model is certain about an identity, the image is likely high quality for that task. This "task-aware" approach, also seen in EQFace by Rushuai Liu, removes the need for subjective human ratings entirely.

Leveraging Vision-Language Priors

The most recent shift involves using Large Multi-modal Models (LMMs) like CLIP, which have already "seen" the entire internet. These models possess a latent understanding of what a "clear" or "blurry" photo looks like because they have associated those concepts with millions of text captions.

Ying Wei introduced LIQE, which treats quality assessment as a vision-language correspondence problem. By asking a model how well an image matches the description "a high-quality photo," researchers can extract zero-shot quality scores. This direction is further refined in GRMP-IQA by Xudong Li, which uses meta-learned soft prompts to adapt these massive models to specific quality tasks with only 20% of the usual training data.

By treating quality as a geometric, statistical, or semantic property rather than a simple label, these researchers are building evaluators that are more robust, more scalable, and ultimately closer to the nuanced way humans perceive the visual world.

Papers in this version

Showing 60 of 166 papers.

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_scaling_perceptual_metrics_beyond_human__434d82bd,
  title = {Scaling Perceptual Metrics Beyond Human Labels},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/scaling-perceptual-metrics-beyond-human-labels/rdr_434d82bd86a1496f848417abda40c790},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_scaling_perceptual_metrics_beyond_human__434d82bd_v1,
  title = {Scaling Perceptual Metrics Beyond Human Labels},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/scaling-perceptual-metrics-beyond-human-labels/rdr_434d82bd86a1496f848417abda40c790/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}