WeCheck: Strong Factual Consistency Checker via Weakly Supervised Learning

Wenhao Wu, Wei Li, Xinyan Xiao, Jiachen Liu, Sujian Li, Yajuan Lyu

Introduction

Large Language Models (LLMs) are increasingly prone to "hallucinations"—generating information that is not supported by the source text. Ensuring factual consistency is a critical challenge for deploying these models in production. However, existing automated checkers often fall short because they are typically trained on synthetic data (e.g., rule-based word swaps) that fails to capture the subtle and diverse errors found in real model-generated text. Furthermore, metrics repurposed from Natural Language Inference (NLI) or Question Answering (QA) tasks often struggle with the specific nuances of text generation.

WeCheck addresses this "seeing is believing" problem by introducing a weakly supervised framework. It aggregates multiple noisy signals to train a highly efficient checker on actual model-generated text, ultimately outperforming models 25 times its size.

Methodology

WeCheck utilizes a two-step pipeline designed to leverage abundant but noisy data generated by LLMs.

The WeCheck framework: Weak Annotation followed by Noise-aware Fine-tuning.

1. Weak Annotation

Instead of relying on expensive human annotations, WeCheck uses existing metrics (such as FactCC, QuestEval, and Summac) as "weak sources." A Generative Labeling Model, based on the Snorkel framework, analyzes the agreement and disagreement between these sources to estimate the latent true label of a sample without requiring ground-truth human labels.

2. Noise-Aware Fine-tuning

To mitigate the risks of training on noisy estimated labels, WeCheck employs two strategies:

  • NLI Warmup: The base model (DeBERTaV3) is first fine-tuned on high-quality NLI data to establish a foundation for logical reasoning.
  • Soft Labels & Filtering: During the final training phase, the model utilizes probabilistic "soft" labels and filters out samples where weak sources show high uncertainty.

Evaluation

The model was trained on bootstrapped samples from various models across multiple tasks: BART for summarization (CNN/DM and XSum), MemNet and dodecaDialogue for dialogue, and PAWS for paraphrase. To evaluate generalizability, the authors used the TRUE benchmark, which aggregates 11 datasets across summarization, dialogue, and paraphrase tasks.

Key Results and Insights

WeCheck achieves state-of-the-art performance, reaching an average ROC AUC of 84.8 on the TRUE benchmark. This represents a 3.3% absolute improvement over previous 11B-parameter models, despite WeCheck having only 435M parameters.

Performance comparison of WeCheck against baseline metrics and large-scale models on the TRUE benchmark.

Key findings include:

  • Efficiency: WeCheck is 30x faster than QA-based methods and 3x faster than sentence-level NLI methods.
  • Robustness to Abstractiveness: Unlike many metrics that degrade as text becomes more "abstractive" (using different phrasing than the source), WeCheck maintains high performance because it is trained on real model outputs.
  • Stability: The model exhibited the lowest variance across different dataset types, indicating its effectiveness as a general-purpose checker.

Conclusion

WeCheck demonstrates that the quality and relevance of the training distribution—specifically using real model-generated data—can be more impactful than raw model size. While the current implementation relies on manually tuned thresholds for weak signals, future work may explore end-to-end training to further minimize the impact of label noise.

Summary updated

BibTeX
@inproceedings{wu2023wecheck,
  author = {Wenhao Wu and Wei Li and Xinyan Xiao and Jiachen Liu and Sujian Li and Yajuan Lyu},
  title = {{WeCheck: Strong Factual Consistency Checker via Weakly Supervised Learning}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2023},
  doi = {10.18653/v1/2023.acl-long.18},
}
Figure