When generative models hallucinate, we often blame their architecture, parameter scale, or decoding strategies. However, a growing body of research suggests a more fundamental culprit: the training data itself.
Standard datasets are frequently riddled with "noisy" targets. For instance, in abstractive summarization, human-written reference summaries often include external context or background knowledge not present in the source text. When a model is trained on these pairs, it doesn't just learn to summarize; it learns that inventing unsupported facts is a valid strategy to minimize training loss. Hallucination, in other words, is a learned behavior.
To break this cycle, a powerful paradigm has emerged across both natural language processing and computer vision: automated training data filtering. By using specialized "judge" models or automated metrics to prune unfaithful, noisy, or misaligned examples before training begins, researchers can force models to prioritize strict grounding and factual consistency.
The Mechanics of Automated Filtering
The core intuition behind automated data filtering is simple: instead of relying on expensive manual curation, we can leverage existing models or targeted metrics to score the alignment between a source input (e.g., a text document, a structured table, or an image) and its corresponding target output (e.g., a summary, a text description, or a caption).
[ Raw Dataset ] ──> [ Automated Judge / Metric ] ──> [ Low-Quality / Unfaithful ] ──> Prune/Mask
│
└──> [ High-Fidelity Subset ] ──> [ Model Training ]
If the target contains unsupported information, logical contradictions, or low-quality noise, the instance is either pruned entirely from the dataset or its problematic tokens are masked out. By training exclusively on this high-fidelity subset, the model learns a clean mapping where every generated token must be strictly justified by the input context.
Cleaning the Textual Pipeline: Summarization and Tables
This filtering approach was heavily pioneered in text-to-text tasks like abstractive summarization and table-to-text generation, where human references are notoriously loose.
In abstractive summarization, Subhajit Chaudhury et al. demonstrated in X-FACTOR: A Cross-metric Evaluation of Factual Correctness in Abstractive Summarization that filtering out article-summary pairs below a specific truthfulness threshold forces models to prioritize faithfulness. Similarly, Nan Feng et al. in Entity-level Factual Consistency of Abstractive Text Summarization and Improving Factual Consistency of Abstractive Summarization via Question Answering showed that standard ground-truth references frequently contain information absent from the source text. By filtering these out using automated scoring metrics—such as question-answering alignment—they prevented models from inheriting the tendency to hallucinate external entities.
Rather than dropping entire examples, fine-grained approaches can target specific errors within an instance. Tanya Goyal et al. in Annotating and Modeling Fine-grained Factuality in Summarization utilized a pre-trained factuality detector to identify and mask non-factual tokens during training, showing that cleaning the token distribution significantly reduces downstream hallucination rates.
This principle extends directly to structured data and specialized domains:
- Table-to-Text Generation: Tianyu Liu et al. (Towards Faithfulness in Open Domain Table-to-text Generation from an Entity-centric View) used entity-centric metrics to select the most factually consistent table-text pairs, reducing hallucinations by up to 60%. Furthermore, Yu Bai et al. (Reasoning Knowledge Filter for Logical Table-to-Text Generation) introduced a reasoning knowledge filter to prune incorrect logical chains, significantly boosting the logical faithfulness of the generated text.
- Knowledge Alignment: The filtering criteria can also be calibrated to the model's own internal knowledge base. Fanqi Wan et al. (Knowledge Verification to Nip Hallucination in the Bud) proposed removing alignment training instances that require factual knowledge the model never acquired during its pretraining phase. This prevents the model from being forced to guess—and thus learning to hallucinate—when faced with unfamiliar topics.
- Specialized RAG: In niche domains like climate science, David Thulke et al. (Listen to the Context: Towards Faithful Large Language Models for Retrieval Augmented Generation on Climate Questions) demonstrated that excluding unfaithful instruction-tuning data drastically improved a model's reliability in Retrieval-Augmented Generation (RAG) tasks, increasing the proportion of supported atomic claims from 30% to 57%. This targeted data curation also enhances a model's "factual adaptiveness"—its ability to adjust its outputs when its internal knowledge conflicts with the input document, as explored by Jongyoon Song et al. in Entity-level Factual Adaptiveness of Fine-tuning based Abstractive Summarization Models.
Extending to the Multimodal Domain: Vision and Language
The challenge of noisy data is amplified in vision-language models (VLMs), where web-crawled image-text pairs are notoriously messy, containing text-heavy advertisements, irrelevant packaging, or low-quality descriptions. The same filtering philosophy applies here: using specialized models to prune the noise.
For long-form image captioning, Monika Wysoczańska et al. developed OVFact: Measuring and Improving Open-Vocabulary Factuality for Long Caption Models, a reference-free evaluation method that uses visual grounding to score caption factuality. By filtering out low-quality, ungrounded captions from large datasets, the authors curated high-quality training subsets that significantly improved the factual precision of downstream VLMs without sacrificing descriptive detail. Similarly, Chen Li et al. (Vision-Language Instruction Tuning: A Review and Analysis) highlighted multi-stage filtering pipelines that leverage specialized models to evaluate semantic alignment and image clarity, ensuring strict usability standards.
In specialized image recognition domains, such as identifying traditional medicinal plants, Yanling Zhang et al. (TCMP-300: A Comprehensive Traditional Chinese Medicinal Plant Dataset for Plant Recognition) utilized CLIP-based automated cleaning to filter out non-representative noise like text-heavy ads or unrelated packaging before human verification, drastically lowering curation costs.
This filtering logic works symmetrically for text-to-image generation and computer vision foundation models:
- Image Generation: Michail Tarasiou et al. (Improving face generation quality and prompt following with synthetic captions) used VLMs to filter large-scale datasets to remove non-human or low-quality images, focusing the model's learning on high-resolution, relevant visual features.
- Dense Prediction: For downstream tasks like canopy height estimation, Daniele Rege Cambrin et al. (Depth Any Canopy: Leveraging Depth Foundation Models for Canopy Height Estimation) integrated a VLM to automatically assess and filter out training imagery containing motion blur or stitching artifacts, ensuring that the fine-tuning process was not degraded by low-fidelity samples.
Why Quality Beats Quantity
The recurring insight across all these domains is a clear rejection of the "more data is always better" maxim. When training data is noisy, scaling up the dataset size simply scales up the model's propensity to hallucinate or ignore its context.
Automated filtering works because it fundamentally alters the optimization landscape. By removing examples where the target text diverges from the source context (or where the image diverges from the caption), the model is never penalized for being faithful, nor is it rewarded for guessing. As demonstrated across these papers, prioritizing high-quality, filtered data over raw volume allows smaller training sets to yield models with superior factual accuracy, better prompt-following capabilities, and significantly fewer hallucinations.
Key Foundational Work
X-FACTOR: A Cross-metric Evaluation of Factual Correctness in Abstractive Summarization (2022) — contributed 2 concept elements to this direction.
Improving Factual Consistency of Abstractive Summarization via Question Answering (2021) — contributed 2 concept elements to this direction.
Annotating and Modeling Fine-grained Factuality in Summarization (2021) — contributed 2 concept elements to this direction.