The Pitfalls of Embedding Dependent Evaluation

127 concepts110 papers1 proposal

Imagine judging a cooking competition not by tasting the food, but by passing each dish through a machine that measures its chemical composition, mapping it to a high-dimensional vector, and calculating its distance to a "perfect" recipe vector. If the machine's sensors happen to be completely blind to sugar, a dessert seasoned with salt instead of sugar might receive a flawless score.

This scenario illustrates the core challenge of modern machine learning evaluation. Because raw data distributions—whether they consist of high-resolution images, complex text, audio waveforms, or molecular graphs—are too high-dimensional to compare directly, we rarely evaluate them in their native space. Instead, we rely on a proxy: we project the data into a latent space using a pre-trained feature extractor (such as InceptionV3, CLIP, or BERT) and compute statistical distances, densities, or diversities within that space.

While this approach makes evaluation computationally tractable, it introduces a dangerous hidden assumption: that the evaluation metric is an objective arbiter of data quality, when it is actually a reflection of the feature extractor’s internal worldview.

How Embedding-Dependent Metrics Distort Reality

To understand why embedding-dependent evaluation can fail, we must look at how we measure the quality and diversity of generative models. In Assessing Generative Models via Precision and Recall, Mario Lucic introduced a framework to disentangle sample quality (precision) from distribution coverage (recall). Similarly, Amey P. Pasarkar et al. expanded dataset diversity metrics via the Vendi Score family to evaluate complex modalities without needing predefined labels.

However, these metrics do not operate on raw pixels or characters; they operate on vectors. As Loïc Simon et al. demonstrate, if the underlying embedding space is blind to specific visual artifacts or data characteristics, the resulting precision and recall metrics will completely fail to catch those deficiencies in the generated output.

This distortion is not merely a theoretical edge case. In On Aliased Resizing and Surprising Subtleties in GAN Evaluation, Gaurav Parmar et al. revealed that standard implementations of metrics like the Fréchet Inception Distance (FID) are highly sensitive to low-level preprocessing choices, such as image resizing libraries, which introduce aliasing artifacts that skew the feature extractor's perception. Furthermore, feature extractors often exhibit inherent biases toward specific textures or object categories, meaning automated metrics can easily diverge from human perception of image quality.

The Circularity of Explanations and Dataset Analysis

The problem deepens when embedding spaces are used to evaluate model interpretability or dataset structure, creating a circular dependency where a model is evaluated using the very representations under scrutiny.

For instance, when evaluating textual counterfactual explanations, Julia El Zini et al. found that faithfulness metrics relying on transformer-based latent representations are highly susceptible to the inherent flaws of the underlying embedding model. If the embedding space misrepresents semantic relationships, the evaluation of the explanation's stability or proximity becomes fundamentally skewed.

A similar constraint appears in clustering and dataset pruning. Yang Young Lu et al. note that explanation frameworks like ACE, which identify explanatory features from an adversarial perspective, are strictly bounded by the initial dimensionality reduction model. If the embedder fails to capture critical structural or biological features, the subsequent explanations are inherently incomplete.

When pruning data to maximize information, as explored by Haoru Tan in Data Pruning by Information Maximization, the selected subset will inevitably inherit and potentially amplify any systematic biases present in the feature extractor used to calculate similarity.

[Raw Data] ---> [Pre-trained Feature Extractor] ---> [Embedding Space] ---> [Evaluation Metric]
                      |                                    |
                      +--- Introduces its own biases ------+
                           and blind spots into the score

Breaking the Closed Loop

Recognizing these dependencies, researchers are actively designing metrics that bypass pre-trained feature extractors entirely or evaluate representations without downstream parameter reliance.

In PQMass: Probabilistic Assessment of the Quality of Generative Models using Probability Mass Estimation, Pablo Lemos et al. propose a method that partitions the raw data space into Voronoi cells and applies a chi-squared test. This eliminates the need for an auxiliary model, making it highly valuable for specialized domains like astrophysics or biology where unbiased pre-trained feature extractors simply do not exist.

Another direction is to scrutinize raw, unparameterized representations directly rather than relying on downstream tasks or fine-tuning as a evaluation crutch. As Heinrich Dinkel et al. demonstrate in the context of audio encoders, evaluating models solely through fine-tuning can mask severe deficiencies in the underlying feature space, as the training process can artificially compensate for poorly organized embeddings.

By understanding that embedding-dependent evaluation is a lens—and often a distorted one—the machine learning community can better diagnose whether a model's failure lies in its outputs or in the tool we use to measure them.

History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_embedding_dependent_evaluation_a062e070,
  title = {embedding-dependent evaluation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/embedding-dependent-evaluation/rdr_a062e07087b6424babd37aaa48186043},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_embedding_dependent_evaluation_a062e070_v1,
  title = {embedding-dependent evaluation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/embedding-dependent-evaluation/rdr_a062e07087b6424babd37aaa48186043/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake