On the Usefulness of Embeddings, Clusters and Strings for Text Generation Evaluation

Tiago Pimentel, Clara Isabel Meister, Ryan CotterellICLR 2023 · OpenReview

Evaluating the quality of AI-generated text is notoriously difficult. While human judgment remains the gold standard, it is slow and expensive. Researchers typically rely on automatic metrics, but common ones like perplexity or BLEU often fail to capture what makes a text "good" or "human-like."

Recently, a metric called MAUVE emerged as a frontrunner, demonstrating a remarkably high correlation with human scores. In a new paper, researchers take a hood-up look at MAUVE to ask a fundamental question: Why does it actually work?

The researchers discover that MAUVE is "right for the wrong reasons." Its success isn't due to its complex new mathematical divergence, but rather its use of embedding-based clusters to represent text. These clusters act as a high-level filter that ignores surface-level noise and focuses on the structural coherence that humans actually care about.

The Problem with String-Based Divergence

Most automatic metrics compare two probability distributions: one from the "true" natural language and one from the AI generator. MAUVE was proposed as a way to measure the "Area Under the Curve" (AUC) of information-theoretic divergences between these two distributions.

However, calculating this over the infinite space of all possible strings is impossible. To make it work in practice, MAUVE’s creators used an approximation: they grouped text samples into clusters based on their embeddings from a pre-trained model (like GPT-2) and then compared the distributions of those clusters.

The authors of this paper wanted to know if MAUVE’s power comes from its sophisticated AUC math or if the clustering "approximation" is actually doing all the heavy lifting.

Experimental Methods

The team conducted a series of controlled experiments to isolate the variables of text evaluation:

  1. String vs. Cluster: They compared "string-based" metrics (using n-gram models to estimate probabilities) against "cluster-based" metrics (using K-means clustering on model embeddings).
  2. Divergence Swap: They tested whether MAUVE’s specific AUC divergence was necessary by replacing it with "classical" measures like Kullback–Leibler (KL) and Jensen-Shannon (JS) divergences, applied to the same clusters.
  3. Probing & Perturbation: To understand what the clusters actually "see," they performed probing (predicting sentiment or topic from cluster IDs) and perturbation (intentionally breaking the text by removing articles or shuffling sentences) to observe how the metrics reacted.

Key Insights: The "Approximation" is the Star

Mathematically, the cluster-based approach is a "bad" approximation of the string-based divergence—it is biased and ignores a massive amount of information. However, this mathematically loose approximation leads to a better metric.

  • Feature Filtering: String-based metrics get bogged down in "surface-level" features (like whether a specific word appears). Cluster-based metrics, by contrast, filter for "discourse-level" features.
  • Coherence over Fluency: The clusters are surprisingly sensitive to coherence (the logical flow between sentences) but relatively indifferent to minor fluency issues (like missing articles).
  • Universal Application: The researchers found that almost any standard divergence measure—even simple ones from the 1940s—performs as well as MAUVE if applied to embedding clusters.

Key Results

Correlation of cluster-based vs. string-based divergences with human judgments of Interestingness.
  • Human Correlation: As shown in the data, cluster-based versions of classical divergences ($\Delta(p_c, q_c)$) correlated just as well—and much more consistently—with human judgments of "Interestingness" and "Sensibility" as the original MAUVE metric did, far outperforming string-based metrics ($\Delta(p_w, q_w)$).
  • The Article Test: In a striking experiment, the researchers removed every "a," "an," and "the" from a text. String-based metrics tanked, but cluster-based metrics barely noticed. In fact, the cluster-based metrics rated the article-free text as better than the baseline output of GPT-2 XL.
  • The Shuffle Test: When the researchers swapped the first and second halves of a text (destroying its logical flow), the cluster-based metrics showed a significant drop, proving they are highly sensitive to high-level document structure.

Limitations and Future Work

The authors acknowledge that while these metrics are powerful, they might be gameable. Because the clusters ignore surface-level features, a model could potentially "cheat" the metric by producing text that has the right high-level "vibe" and coherence but contains many small grammatical errors.

Additionally, the study was limited to English. Future work is needed to see if embedding clusters capture the same useful abstractions in morphologically rich languages or different scripts. The authors suggest that the next frontier is finding even better ways to "discretize" the embedding space to align with specific linguistic attributes we want to measure.

Summary updated

Abstract

A good automatic evaluation metric for language generation ideally correlates highly with human judgements of text quality. Yet, there is a dearth of such metrics, which inhibits the rapid and efficient progress of language generators. One exception is the recently proposed Mauve. In theory, Mauve measures an information-theoretic divergence between two probability distributions over strings: one representing the language generator under evaluation; the other representing the true natural language distribution. Mauve's authors argue that its success comes from the qualitative properties of their proposed divergence. Yet in practice, as this divergence is uncomputable, Mauve approximates it by measuring the divergence between multinomial distributions over clusters instead, where cluster assignments are attained by grouping strings based on a pretrained language model's embeddings. As we show, however, this is not a tight approximation---in either theory or practice. This begs the question: why does Mauve work so well? In this work, we show that \mauve was right for the wrong reasons, and that its newly proposed divergence is not necessary for its high performance. In fact, classical divergences paired with its proposed cluster-based approximation may actually serve as better evaluation metrics. We finish the paper with a probing analysis; this analysis leads us to conclude that---by encoding syntactic- and coherence-level features of text, while ignoring surface-level features---such cluster-based approximations to string distributions may simply be better for evaluating state-of-the-art language generators.
BibTeX
@inproceedings{pimentel2023usefulness,
  author = {Tiago Pimentel and Clara Isabel Meister and Ryan Cotterell},
  title = {{On the Usefulness of Embeddings, Clusters and Strings for Text Generation Evaluation}},
  booktitle = {International Conference on Learning Representations},
  year = {2023},
  url = {https://openreview.net/forum?id=bvpkw7UIRdU},
}
Figure