Representational Constraints of Embedding Based Metrics

This direction is split into other directions: embedding-dependent evaluation. Its last version stays readable and citable here.

Version 1 of 1199 papers213 conceptsfrozen 2026-10-04

Modern machine learning has largely moved away from evaluating models using simple surface-level statistics like word overlap (BLEU) or pixel-wise distance. Instead, we rely on embedding-based metrics—tools that project complex data into a latent vector space to measure semantic similarity, diversity, and quality.

While these metrics align much better with human judgment, a growing body of research reveals a fundamental "representational bottleneck." As Michael Stenger and colleagues note in Evaluation is key: a survey on evaluation measures for synthetic time series, these metrics introduce a "black box" dependency: the evaluation is only as reliable as the underlying encoder used to process the data.

The Latent Mapping Problem

The core intuition behind embedding-based evaluation is that a well-trained encoder (like BERT or CLIP) can "understand" the essence of data. By calculating the distance between two vectors, we hope to measure how similar two concepts are. However, this mapping is inherently lossy.

If an encoder is blind to a specific feature—such as the tempo of a song or the structural nuances of a protein—any metric built on top of it will also be blind to those features. Andrea Agostinelli and the team at Google DeepMind demonstrated this in Diversity-Rewarded CFG Distillation, finding that diversity rewards in music generation are strictly limited by what the embedding model can actually "see." If the embedding space doesn't capture artistic style, the training process cannot encourage variation in that style.

Blind Spots in Diversity and Quality

This representational limit creates significant risks when measuring the health of generative models. Popular metrics like MAUVE, developed by Krishna Pillutla and others, use quantized embedding spaces to detect "mode collapse" (where a model repeats the same output).

However, as Yuchang Zhu argues in Measuring Diversity in Synthetic Datasets, even mathematically rigorous metrics like DCScore are fundamentally constrained by the encoder. If the encoder fails to capture semantic nuances, the metric will fail to penalize a model for missing those nuances. This is echoed in the work of Pum Jun Kim on STREAM, where the accuracy of video quality assessment is strictly bounded by the representational capacity of the spatial-temporal feature extractor.

The Erasure Illusion and Semantic Drift

Perhaps the most surprising failure mode is what Hengrui Jia calls The Erasure Illusion. When we try to make a model "forget" specific sensitive information, we often use embedding distances to verify the unlearning.

Jia's research shows that models can pass these tests while still retaining the knowledge. This happens because the same information, when paraphrased, shifts to a different part of the vector space. Because the metric relies on a specific geometric neighborhood, it falsely indicates that the information is gone, when in reality, the "lens" of the embedding model simply lost focus.

Inherited Biases and Structural Disparities

Because these metrics rely on pre-trained models, they inherit the biases of their parents. In Estimating Structural Disparities for Face Models, Shervin Ardeshir points out that if an embedding model is insensitive to certain protected characteristics, it will be unable to detect unfairness related to those characteristics.

Similarly, A. Sahoo found in Quantitative LLM Judges that while LLMs can explain their reasoning, their ability to assign consistent numeric scores is limited by the "semantic richness" of their internal representations. If the base model's world-view is flawed, the secondary scoring layer cannot produce a reliable evaluation.

Moving Beyond the Bottleneck

The research direction across these 169 papers suggests that we cannot treat embeddings as neutral ground truth. Whether it is Quentin Garrido using the "effective rank" of embeddings in RankMe or Robert Geirhos exploring visual memory, the field is moving toward a more critical understanding of latent spaces.

To build better models, we must first acknowledge that our metrics are not measuring the data itself, but rather a specific, often biased, and always limited interpretation of that data. The "evaluation gap" can only be closed by developing encoders that are as diverse and nuanced as the tasks we ask them to judge.

Papers in this version

Showing 60 of 199 papers.

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_representational_constraints_of_embeddin_aedba6be,
  title = {Representational Constraints of Embedding Based Metrics},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/representational-constraints-of-embedding-based-metrics/rdr_aedba6be11654789a8c41003c4a7768b},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_representational_constraints_of_embeddin_aedba6be_v1,
  title = {Representational Constraints of Embedding Based Metrics},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/representational-constraints-of-embedding-based-metrics/rdr_aedba6be11654789a8c41003c4a7768b/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}