When evaluating natural language generation (NLG) and multimodal systems, the standard practice has long been to compare a model's output against a human-written "gold standard" reference text using token-overlap metrics like BLEU or ROUGE. This approach operates on an implicit assumption: that there is a single, definitive "correct" answer for a given prompt or source document.
However, open-ended tasks like text summarization, grammatical error correction (GEC), and image captioning exhibit a one-to-many property. For any given input, there are countless valid ways to phrase an output, select salient facts, or correct a sentence. When an evaluation metric forces a model to align with a single human reference, it introduces reference bias—unfairly penalizing high-quality, fluent, and accurate outputs simply because they choose a different stylistic path or focus on a different set of valid details than the individual human annotator did.
[ Source Document / Input ]
|
+---------------------+---------------------+
| |
[ Valid Output A ] [ Valid Output B (Gold Reference) ]
(Different phrasing/facts) (Specific annotator style/choices)
| |
+-----------------+-------------------------+
|
[ Token Overlap Metric ]
|
Low Score (Artificially Penalized)
To build more accurate evaluators, researchers are shifting away from rigid token-matching against a single reference. Instead, the field is moving toward multi-reference aggregation, semantic fact extraction, and reference-less evaluation.
The Cost of Strict Imitation
The consequences of reference bias are not limited to misleading leaderboards; they actively distort how models are trained. In Grammatical Error Correction, Christopher Bryant et al. demonstrated that evaluating against a single reference artificially deflates system scores by ignoring equally valid alternative corrections, masking the true proficiency of top-performing models. When models are trained via standard maximum likelihood estimation to maximize overlap with these narrow references, they are forced to strictly imitate a specific annotator's style rather than learning general linguistic quality.
To break this dependency during training, the authors of the BRIO paradigm (Khang Nhut Lam et al.) introduced contrastive learning to rank multiple candidate summaries based on holistic quality rather than forcing token-level imitation. Similarly, Sangwon Ryu et al. developed a Multi-Dimensional Optimization framework that uses reinforcement learning to balance competing traits like coherence, consistency, and fluency, allowing models to generate concise, human-preferred summaries that diverge from the exact wording of the gold standard.
Deconstructing the Reference: Multi-Reference and Structural Approaches
If a reference text must be used, one way to mitigate bias is to capture the consensus of multiple human perspectives rather than relying on a single author. The Pyramid Method, introduced by Ani Nenkova et al., addresses content selection by extracting a weighted inventory of "Summary Content Units" from multiple human summaries. This ensures that a system summary is rewarded for selecting any set of facts deemed important by a human consensus, rather than being penalized for missing the specific facts chosen by a single person.
To make this process scalable and reproducible, Mohit Bansal et al. developed QAPyramid, a framework that breaks down reference summaries into fine-grained question-answer pairs based on linguistic structures. This allows the evaluation to award partial credit for semantic alignment without being derailed by stylistic variations.
Taking this a step further, Hardy et al. bypassed full-text reference summaries entirely with HighRES. Instead of writing a summary, human annotators highlight salient text directly within the source document. These highlights form a reference-free saliency "heat map" that evaluates a model's content selection directly against the source, completely avoiding the stylistic bias introduced by a written reference.
Traditional: [System Summary] <--- Token Match ---> [Single Human Reference Summary] (Biased)
QAPyramid: [System Summary] <--- QA Verification ---> [Atomic QA Pairs from References] (Granular)
HighRES: [System Summary] <--- Saliency Match ---> [Source Document Highlights] (No Reference Text)
Evaluating Semantics and Facts Over Tokens
Another major lever for eliminating reference bias is shifting the evaluation metric from how something is said (tokens) to what is said (semantics). Standard overlap metrics like ROUGE penalize abstractive models for paraphrasing. To resolve this, Xinnuo Xu et al. proposed Corr-F and Corr-A, metrics that extract semantic facts from texts using Semantic Role Labeling. By matching facts rather than words, these metrics align much more closely with human judgment and reward valid paraphrases.
Similarly, Matan Eyal et al. introduced APES (Answering Performance for Evaluation of Summaries). APES automatically generates question-answer pairs from the source text's key entities and evaluates a summary based on whether it retains enough information to answer those questions, turning evaluation into a functional task of information preservation.
Going Reference-Less: Intrinsic Quality and LLM Judges
The ultimate solution to reference bias is to remove the reference text entirely, evaluating the generated output directly against the input or via intrinsic quality dimensions. In GEC, Courtney Napoles et al. demonstrated that reference-less "grammaticality" metrics—which evaluate the fluency and correctness of the output text directly—can achieve higher correlation with human judgments than traditional metrics that measure overlap with a human correction.
With the advent of powerful large language models, reference-free evaluation has expanded into complex multimodal and creative domains:
- Multimodal Evaluation: For image and video captioning, Tony Cheng Tong et al. developed G-VEval. By leveraging GPT-4o's multimodal capabilities and Chain-of-Thought reasoning, it evaluates descriptions directly against the visual media, bypassing incomplete or biased human reference texts that often omit valid visual details.
- Utility-Based Metrics: Instead of measuring how similar an output is to a target, metrics can measure how much work it saves a human. Yongqiang Ma et al. introduced Revision Distance ($D_{Revision}$), which uses an LLM as a proxy user to generate structured, self-explanatory revision suggestions, measuring quality by the edit effort required to reach a high standard rather than token divergence.
The Flawed Foundation of Meta-Evaluation
The distortion caused by reference bias extends to how we evaluate our evaluation metrics (meta-evaluation). Xiang Dai 0001 et al. revealed that providing human annotators with a reference summary introduces a strong cognitive bias, causing them to penalize high-quality candidate summaries that use a different structure or phrasing than the provided reference. This means that reference-mediated human evaluation fails to provide an objective ground truth for benchmarking automated metrics.
Furthermore, the human-written "gold standards" themselves are often deeply flawed:
- Noisy Baselines: In code intelligence tasks, Kang Yang et al. discovered that human-written code comments in popular pre-training datasets are frequently noisy, outdated, or incomplete. Replacing these flawed human references with LLM-generated synthetic comments actually yielded superior training data and fairer evaluation benchmarks.
- Metric Favoritism: When automated metrics are tuned against biased human ground truths, they inherit those systematic skews. Pius von Däniken et al. designed the Favi-Score to diagnose this exact phenomenon, quantifying how much an evaluation metric systematically favors specific models due to biases embedded within the reference data.
- Domain-Specific Filtering: In specialized tasks like Video Anomaly Understanding, standard metrics get bogged down by stylistic "fluff" and normal background content. To combat this, Jiaxu Leng et al. introduced CG-CoE, a class-guided chain-of-evaluation metric that isolates and evaluates only the core anomalous events, ensuring the metric is robust against variations in human description styles.
By moving away from the assumption that a single human text represents the sole ground truth, these diverse methodologies allow evaluation metrics to judge generative models on their actual merits—accuracy, fluency, and utility—rather than their ability to mimic a single annotator's stylistic choices.
Key Foundational Work
How Far are We from Fully Automatic High Quality Grammatical Error Correction? (2015) — contributed 1 concept elements to this direction.
Evaluating Content Selection in Summarization: The Pyramid Method (2004) — contributed 1 concept elements to this direction.