Introduction
Large Language Models (LLMs) are increasingly prone to "hallucinations"—generating information that is not supported by the source text. Ensuring factual consistency is a critical challenge for deploying these models in production. However, existing automated checkers often fall short because they are typically trained on synthetic data (e.g., rule-based word swaps) that fails to capture the subtle and diverse errors found in real model-generated text. Furthermore, metrics repurposed from Natural Language Inference (NLI) or Question Answering (QA) tasks often struggle with the specific nuances of text generation.
WeCheck addresses this "seeing is believing" problem by introducing a weakly supervised framework. It aggregates multiple noisy signals to train a highly efficient checker on actual model-generated text, ultimately outperforming models 25 times its size.
Methodology
WeCheck utilizes a two-step pipeline designed to leverage abundant but noisy data generated by LLMs.
1. Weak Annotation
Instead of relying on expensive human annotations, WeCheck uses existing metrics (such as FactCC, QuestEval, and Summac) as "weak sources." A Generative Labeling Model, based on the Snorkel framework, analyzes the agreement and disagreement between these sources to estimate the latent true label of a sample without requiring ground-truth human labels.
2. Noise-Aware Fine-tuning
To mitigate the risks of training on noisy estimated labels, WeCheck employs two strategies:
- NLI Warmup: The base model (DeBERTaV3) is first fine-tuned on high-quality NLI data to establish a foundation for logical reasoning.
- Soft Labels & Filtering: During the final training phase, the model utilizes probabilistic "soft" labels and filters out samples where weak sources show high uncertainty.
Evaluation
The model was trained on bootstrapped samples from various models across multiple tasks: BART for summarization (CNN/DM and XSum), MemNet and dodecaDialogue for dialogue, and PAWS for paraphrase. To evaluate generalizability, the authors used the TRUE benchmark, which aggregates 11 datasets across summarization, dialogue, and paraphrase tasks.
Key Results and Insights
WeCheck achieves state-of-the-art performance, reaching an average ROC AUC of 84.8 on the TRUE benchmark. This represents a 3.3% absolute improvement over previous 11B-parameter models, despite WeCheck having only 435M parameters.
Key findings include:
- Efficiency: WeCheck is 30x faster than QA-based methods and 3x faster than sentence-level NLI methods.
- Robustness to Abstractiveness: Unlike many metrics that degrade as text becomes more "abstractive" (using different phrasing than the source), WeCheck maintains high performance because it is trained on real model outputs.
- Stability: The model exhibited the lowest variance across different dataset types, indicating its effectiveness as a general-purpose checker.
Conclusion
WeCheck demonstrates that the quality and relevance of the training distribution—specifically using real model-generated data—can be more impactful than raw model size. While the current implementation relies on manually tuned thresholds for weak signals, future work may explore end-to-end training to further minimize the impact of label noise.
Summary updated