When a large language model summarizes a document, answers a question based on a text corpus, or generates a report, it faces a dual challenge: it must be both linguistically fluent and factually accurate. Historically, evaluating these models relied on reference-based metrics like ROUGE or BLEU, which measure how closely a generated text matches a human-written reference.
However, reference-based metrics suffer from a critical flaw: a generated summary can be highly fluent, share many words with a human reference, and yet introduce a single fabricated word that completely distorts the facts. To solve this, researchers have shifted toward source-grounded evaluation. Instead of measuring stylistic similarity to a human proxy, source-grounded evaluation treats the original source document as the absolute ground truth, verifying whether the generated claims are mathematically and logically entailed by the source material.
Question Answering as a Fact Checking Proxy
How do you automatically determine if a generated summary is faithful to its source without requiring a human to read both? One of the most intuitive mechanisms treats evaluation as a reading comprehension task using automated Question Answering (QA).
The core idea is simple: if a generated text contains the same factual information as the source document, then questions written based on the generated text should yield identical answers when answered using the source document.
[Generated Text] ──► (Question Generation) ──► [Factual Questions]
│
┌──────────┴──────────┐
▼ ▼
(Answer via Summary) (Answer via Source)
│ │
└──────────┬──────────┘
▼
[Compare for Alignment]
Esin Durmus et al. formalized this in the FEQA framework (FEQA: A Question Answering Evaluation Framework for Faithfulness Assessment in Abstractive Summarization). The system extracts key entities from a generated summary, uses a question-generation model to create queries around those entities, and then tasks a QA model with finding the answers in the original source text. If the source text cannot answer the question or provides a conflicting answer, a hallucination is detected.
Conversely, evaluation can be anchored directly in the source text. Saadia Gabriel et al. demonstrated in the GO FIGURE framework (GO FIGURE: A Meta Evaluation of Factuality in Summarization) that generating questions from the source document and verifying them against the summary tends to be more robust across diverse domains. By shifting the baseline from human-written references to automated source-grounded QA, as also explored by Luyang Huang et al. in Efficient Attentions for Long Document Summarization, evaluation frameworks can directly measure factual deviations and pinpoint exactly where a model began to hallucinate.
Enforcing Verifiability via Provenance and Citations
A more stringent way to ensure source grounding is to force the generation model to "show its work." Instead of allowing a model to output a standalone claim, strict evaluation frameworks require the model to provide provenance—the exact sentence, text span, or data cell that supports the assertion.
This mechanism transforms open-ended fact-checking into a structured verification problem. James Thorne et al. pioneered this approach with the FEVER shared task (The Fact Extraction and VERification (FEVER) Shared Task), establishing a strict evaluation framework where a system's verdict is only considered correct if it also retrieves the exact sentence-level evidence from Wikipedia. This prevents models from guessing correctly based on statistical biases or internal parametric memory, a paradigm later scaled across multiple knowledge-intensive tasks by Aleksandra Piktus et al. in the KILT benchmark (KILT: a Benchmark for Knowledge Intensive Language Tasks).
When applied to long-form generation, this constraint acts as an automated audit trail. For example, the ARGUE framework introduced by James Mayfield et al. (On the Evaluation of Machine-Generated Reports) breaks down reports into information nuggets and evaluates them by checking explicit citations for relevance and support against a closed corpus.
This verification mechanism can also be integrated directly into the model's generation process to prevent errors before they happen:
- Tabular Grounding: Jinyoung Yeo et al. developed the Question-then-Pinpoint (QtP) framework (Unveiling Implicit Table Knowledge with Question-Then-Pinpoint Reasoner for Insightful Table Summarization), forcing models to pinpoint specific data cells as evidence before deriving a high-level narrative insight.
- Textual Grounding: Sanwoo Lee et al. utilized a conversational "quote-then-score" approach in Unleashing Large Language Models' Proficiency in Zero-shot Essay Scoring, requiring the model to extract exact textual quotes before assigning a numerical evaluation.
Lexical and Structural Alignment
While QA and citation checking handle complex semantic reasoning, fabrications can often be caught using simpler lexical and structural constraints. Extrinsic hallucinations—introducing facts out of thin air—frequently manifest as new named entities (such as specific people, places, or dates) that never existed in the source text.
The authors of Entity-level Factual Consistency of Abstractive Text Summarization, Nan Feng et al., quantified this behavior by measuring entity coverage. By calculating the exact ratio of named entities in the generated output that are present in the original source document, they created a precise metric for identifying when a model is inventing information. This concept was extended by Haopeng Zhang et al. (Improving the Faithfulness of Abstractive Summarization via Entity Coverage Control), who used these entity ratios as control codes to steer models away from relying on their internal pre-trained knowledge.
For highly verbose instruction-following models, token-level precision metrics serve a similar purpose. Vaibhav Adlakha et al. proposed metrics like K-Precision (Evaluating Correctness and Faithfulness of Instruction-Following Models for Question Answering) to measure the exact percentage of generated tokens that can be found in the provided context, successfully separating context-grounded responses from ungrounded fluff.
Taking structural alignment to its mathematical limit, Nico Daheim et al. framed source grounding as an information reconstruction problem in Controllable Factuality in Document-Grounded Dialog Systems Using a Noisy Channel Model. Under this framework, a generated response is faithful if a channel model can successfully reconstruct the original source document from it. If the response contains hallucinations or ungrounded claims, the mathematical probability of recovering the original text drops, providing a clean, reference-free proxy for factual entailment.
Key Foundational Work
FEVER: a Large-scale Dataset for Fact Extraction and VERification (2018) — contributed 1 concept elements to this direction.
The Fact Extraction and VERification (FEVER) Shared Task (2018) — contributed 1 concept elements to this direction.
Handling Divergent Reference Texts when Evaluating Table-to-Text Generation. (2019) — contributed 1 concept elements to this direction.