Near-Negative Distinction: Giving a Second Life to Human Evaluation Datasets

Philippe Laban, Chien-Sheng Wu, Wenhao Liu, Caiming Xiong

Overview

This paper introduces Near-Negative Distinction (NND), a framework that transforms expensive, one-off human evaluation datasets into reusable automatic benchmarks. By testing whether a model assigns a higher likelihood to a high-quality output than to a "near-negative" (an output with a known human-labeled error), NND provides a granular, highly-correlated alternative to standard metrics like BLEU or ROUGE.

The Challenge of Human Evaluation

In Natural Language Generation (NLG), human evaluation is the "gold standard," but it is notoriously expensive, slow, and difficult to reproduce. Once a study is finished, the resulting data—often containing thousands of expert-labeled errors—is rarely used again.

Conversely, automatic metrics like BLEU or ROUGE are cheap but flawed. They rely on n-gram overlap with a reference text, which often fails to capture semantic meaning or factual correctness. Furthermore, these metrics provide a single aggregate score that doesn't explain why a model is failing. The authors seek to bridge this gap by turning "dead" human annotation data into a "live" diagnostic tool.

The NND Framework

The NND framework operates in three main stages:

  1. Dataset Repurposing: Existing human evaluation datasets, where multiple model outputs were graded for a single input, are grouped. Candidates are assigned quality levels based on human labels (e.g., "No Error" vs. "Not Fluent").
  2. Creating NND Tests: Pairs are formed consisting of one high-quality candidate ($C_{high}$) and one "near-negative" candidate ($C_{low}$) containing a specific error.
  3. Model Administration: To evaluate a new model, researchers ask it to compute the log-likelihood of both candidates. A model "passes" the test if it assigns a higher probability to the high-quality text than the erroneous one.
The NND framework: from human annotations to model likelihood tests.

Crucially, the likelihood is normalized by sequence length to prevent the model from simply preferring shorter or longer sentences. This allows for an Overall Pass Rate and specific Error Category Pass Rates.

Datasets and Validation

The authors validated NND across three diverse NLG tasks:

  • Question Generation (QGen): The Quiz Design (QD) dataset, featuring 3,164 questions annotated by teachers.
  • Generative Question Answering (QA): Challenge 300, a suite of 300 difficult questions across five categories.
  • Summarization: SummEval (general quality attributes) and FRANK (focused on factual consistency).

Key Insights

  • Likelihood as a Diagnostic Probe: NND treats a model's internal probability distribution as a reflection of its "preferences." If a model thinks a hallucinated summary is more likely than a factual one, NND flags a specific deficiency in training.
  • Reference-less Evaluation: Unlike standard metrics, NND doesn't need a "gold" human reference. It only needs to know which of two model-generated strings is better, making it more flexible for open-ended tasks.
  • The "Hallucination" Trade-off: By applying NND during training, the authors found that as models train longer, coherence and relevance improve, but the ability to distinguish factual from non-factual statements often decreases, suggesting standard objectives may encourage "fluent hallucination."

Results and Performance

NND consistently outperformed standard metrics (BLEU, METEOR, ROUGE) in correlating with human judgments. In Question Generation, NND achieved a Kendall rank correlation ($\tau$) of 0.78, compared to 0.65 for BLEU.

NND Pass Rates for BART-Large and PEGASUS across SummEval and FRANK benchmarks.

NND also allowed for a head-to-head comparison of BART and PEGASUS. While they had nearly identical ROUGE scores, NND revealed that PEGASUS was significantly better at avoiding factual consistency errors (Verifiability) and maintaining consistency.

Limitations

  • Access to Likelihoods: NND requires access to internal token probabilities, making it difficult to use with "black-box" APIs like GPT-4.
  • Calibration Dependency: The method assumes the model is well-calibrated; a poorly calibrated model might assign high likelihoods to gibberish.
  • Static Error Sets: NND is limited by the errors found in the original human study. Future work could involve "evolving" NND sets by adding new annotations to capture failure modes of newer models.

Summary updated

Abstract

Precisely assessing the progress in natural language generation (NLG) tasks is challenging, and human evaluation to establish a preference in a model's output over another is often necessary. However, human evaluation is usually costly, difficult to reproduce, and non-reusable. In this paper, we propose a new and simple automatic evaluation method for NLG called Near-Negative Distinction (NND) that repurposes prior human annotations into NND tests. In an NND test, an NLG model must place a higher likelihood on a high-quality output candidate than on a near-negative candidate with a known error. Model performance is established by the number of NND tests a model passes, as well as the distribution over task-specific errors the model fails on. Through experiments on three NLG tasks (question generation, question answering, and summarization), we show that NND achieves a higher correlation with human judgments than standard NLG evaluation metrics. We then illustrate NND evaluation in four practical scenarios, for example performing fine-grain model analysis, or studying model training dynamics. Our findings suggest that NND can give a second life to human annotations and provide low-cost NLG evaluation.
BibTeX
@inproceedings{laban2022near,
  author = {Philippe Laban and Chien-Sheng Wu and Wenhao Liu and Caiming Xiong},
  title = {{Near-Negative Distinction: Giving a Second Life to Human Evaluation Datasets}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2022},
  doi = {10.18653/v1/2022.emnlp-main.135},
}
Figure