Fine-Grained Self-Endorsement Improves Factuality and Reasoning

Ante Wang, Linfeng Song, Baolin Peng, Ye Tian, Lifeng Jin, Haitao Mi, Jinsong Su, Dong YuarXiv 2024 · DOI · arXiv

Introduction

Large Language Models (LLMs) have demonstrated remarkable capabilities across diverse domains, yet they remain prone to "hallucinations"—the generation of factually incorrect but plausible-sounding information. This issue is particularly acute in knowledge-intensive tasks such as long-form biography generation or complex multi-step reasoning.

Current mitigation strategies generally fall into two categories: ensemble methods (e.g., Self-Consistency) and self-refinement (e.g., Chain-of-Verification). However, ensemble methods often operate at a coarse, response-level granularity, making them ineffective for long-form text where errors may be sparse. Conversely, self-refinement often necessitates a high-capacity "teacher" model to provide accurate critiques, limiting its utility for smaller, open-source models.

The Self-Endorsement Framework

To address these limitations, researchers have developed a "Self-Endorsement" framework that leverages the internal consistency of an LLM at a granular, fact-level scale. The process consists of four distinct stages:

  1. Candidate Sampling: The model generates $N$ diverse responses to a single prompt using nucleus sampling.
  2. Fact Decomposition: Each response is parsed into atomic, self-contained factual statements.
  3. Fact Verification: This core stage involves "cross-response validation." Each atomic fact from one sample is verified against the content of all other samples. This produces an "Endorsement Score" for every claim, representing a peer-review consensus within the model's own output space.
  4. Final Response Production: The framework offers two paths for final output: Selection, where the original candidate with the highest average endorsement score is chosen, or Regenerative Synthesis, where a new response is constructed using only the facts that exceeded a specific endorsement threshold.
The Self-Endorsement pipeline: from initial sampling and fact decomposition to cross-verification and final response production.

To maintain computational efficiency, the framework employs Context Pruning via the BM25 algorithm. This ensures that during the verification phase, the model only processes the most relevant segments of other responses, reducing the token overhead by approximately 50%.

Experimental Evaluation

The framework was rigorously tested across three benchmarks: Biographies (long-form factuality), TriviaQA (short-form factuality), and GSM8K (mathematical reasoning). Experiments primarily utilized LLaMA2 (7B and 70B) and Mixtral-8x7B models.

Key Results

  • Factuality Gains: On the Biographies task, the LLaMA2-7B model saw its factual accuracy rise from 53.2% to 67.7% through regenerative synthesis.
  • Granular vs. Coarse Voting: The research demonstrates that "the wisdom of the crowd" is most effective when applied at the fact level. While a model might err on a specific detail in one sample, it often provides the correct information in others. Fact-level voting allows the system to salvage correct information from otherwise flawed responses.
  • Reasoning and Scale: The method consistently outperformed standard Self-Consistency and Chain-of-Verification (CoVe) baselines. On the GSM8K dataset, it improved both final answer accuracy and the quality of intermediate reasoning steps.
Selection of the most consistent response based on majority consensus across multiple samples.

Discussion and Conclusion

A significant finding of this work is the strong positive correlation between a claim's Endorsement Score and its objective truth. Furthermore, the framework proves particularly beneficial for smaller models (e.g., LLaMA2-7B), providing a pathway for improving open-source AI performance without relying on massive teacher models.

While the computational cost of multiple verification calls remains a challenge, the use of context pruning and the potential for future distillation into base models suggest that fine-grained self-endorsement is a robust strategy for enhancing the reliability of LLMs. Future research will focus on optimizing the verification phase, potentially using specialized encoders to further reduce latency.

Summary updated

Abstract

This work studies improving large language model (LLM) generations at inference time by mitigating fact-conflicting hallucinations. Particularly, we propose a self-endorsement framework that leverages the fine-grained fact-level comparisons across multiple sampled responses. Compared with prior ensemble methods (Wang et al., 2022;Chen et al., 2023)) that perform response-level selection, our approach can better alleviate hallucinations, especially for longform generation tasks. Our approach can broadly benefit smaller and open-source LLMs as it mainly conducts simple content-based comparisons. Experiments on Biographies show that our method can effectively improve the factuality of generations with simple and intuitive prompts across different scales of LLMs. Besides, comprehensive analyses on TriviaQA and GSM8K demonstrate the potential of self-endorsement for broader application.
BibTeX
@misc{wang2024fine,
  author = {Ante Wang and Linfeng Song and Baolin Peng and Ye Tian and Lifeng Jin and Haitao Mi and Jinsong Su and Dong Yu},
  title = {{Fine-Grained Self-Endorsement Improves Factuality and Reasoning}},
  year = {2024},
  eprint = {2402.15631},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2402.15631},
}
Figure