Training Data Filtering for Factual and Faithful Generation

22 concepts14 papers

When generative models hallucinate, we often blame their architecture, parameter scale, or decoding strategies. However, a growing body of research suggests a more fundamental culprit: the training data itself.

Standard datasets are frequently riddled with "noisy" targets. For instance, in abstractive summarization, human-written reference summaries often include external context or background knowledge not present in the source text. When a model is trained on these pairs, it doesn't just learn to summarize; it learns that inventing unsupported facts is a valid strategy to minimize training loss. Hallucination, in other words, is a learned behavior.

To break this cycle, a powerful paradigm has emerged across both natural language processing and computer vision: automated training data filtering. By using specialized "judge" models or automated metrics to prune unfaithful, noisy, or misaligned examples before training begins, researchers can force models to prioritize strict grounding and factual consistency.

The Mechanics of Automated Filtering

The core intuition behind automated data filtering is simple: instead of relying on expensive manual curation, we can leverage existing models or targeted metrics to score the alignment between a source input (e.g., a text document, a structured table, or an image) and its corresponding target output (e.g., a summary, a text description, or a caption).

[ Raw Dataset ] ──> [ Automated Judge / Metric ] ──> [ Low-Quality / Unfaithful ] ──> Prune/Mask
                             │
                             └──> [ High-Fidelity Subset ] ──> [ Model Training ]

If the target contains unsupported information, logical contradictions, or low-quality noise, the instance is either pruned entirely from the dataset or its problematic tokens are masked out. By training exclusively on this high-fidelity subset, the model learns a clean mapping where every generated token must be strictly justified by the input context.

Cleaning the Textual Pipeline: Summarization and Tables

This filtering approach was heavily pioneered in text-to-text tasks like abstractive summarization and table-to-text generation, where human references are notoriously loose.

In abstractive summarization, Subhajit Chaudhury et al. demonstrated in X-FACTOR: A Cross-metric Evaluation of Factual Correctness in Abstractive Summarization that filtering out article-summary pairs below a specific truthfulness threshold forces models to prioritize faithfulness. Similarly, Nan Feng et al. in Entity-level Factual Consistency of Abstractive Text Summarization and Improving Factual Consistency of Abstractive Summarization via Question Answering showed that standard ground-truth references frequently contain information absent from the source text. By filtering these out using automated scoring metrics—such as question-answering alignment—they prevented models from inheriting the tendency to hallucinate external entities.

Rather than dropping entire examples, fine-grained approaches can target specific errors within an instance. Tanya Goyal et al. in Annotating and Modeling Fine-grained Factuality in Summarization utilized a pre-trained factuality detector to identify and mask non-factual tokens during training, showing that cleaning the token distribution significantly reduces downstream hallucination rates.

This principle extends directly to structured data and specialized domains:

Extending to the Multimodal Domain: Vision and Language

The challenge of noisy data is amplified in vision-language models (VLMs), where web-crawled image-text pairs are notoriously messy, containing text-heavy advertisements, irrelevant packaging, or low-quality descriptions. The same filtering philosophy applies here: using specialized models to prune the noise.

For long-form image captioning, Monika Wysoczańska et al. developed OVFact: Measuring and Improving Open-Vocabulary Factuality for Long Caption Models, a reference-free evaluation method that uses visual grounding to score caption factuality. By filtering out low-quality, ungrounded captions from large datasets, the authors curated high-quality training subsets that significantly improved the factual precision of downstream VLMs without sacrificing descriptive detail. Similarly, Chen Li et al. (Vision-Language Instruction Tuning: A Review and Analysis) highlighted multi-stage filtering pipelines that leverage specialized models to evaluate semantic alignment and image clarity, ensuring strict usability standards.

In specialized image recognition domains, such as identifying traditional medicinal plants, Yanling Zhang et al. (TCMP-300: A Comprehensive Traditional Chinese Medicinal Plant Dataset for Plant Recognition) utilized CLIP-based automated cleaning to filter out non-representative noise like text-heavy ads or unrelated packaging before human verification, drastically lowering curation costs.

This filtering logic works symmetrically for text-to-image generation and computer vision foundation models:

Why Quality Beats Quantity

The recurring insight across all these domains is a clear rejection of the "more data is always better" maxim. When training data is noisy, scaling up the dataset size simply scales up the model's propensity to hallucinate or ignore its context.

Automated filtering works because it fundamentally alters the optimization landscape. By removing examples where the target text diverges from the source context (or where the image diverges from the caption), the model is never penalized for being faithful, nor is it rewarded for guessing. As demonstrated across these papers, prioritizing high-quality, filtered data over raw volume allows smaller training sets to yield models with superior factual accuracy, better prompt-following capabilities, and significantly fewer hallucinations.

Key Foundational Work

X-FACTOR: A Cross-metric Evaluation of Factual Correctness in Abstractive Summarization (2022) — contributed 2 concept elements to this direction.
Improving Factual Consistency of Abstractive Summarization via Question Answering (2021) — contributed 2 concept elements to this direction.
Annotating and Modeling Fine-grained Factuality in Summarization (2021) — contributed 2 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_vision_language_data_curation_f6589fad,
  title = {vision-language data curation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/vision-language-data-curation/rdr_f6589fadc70243b082688b2b2946aec0},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_vision_language_data_curation_f6589fad_v1,
  title = {vision-language data curation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/vision-language-data-curation/rdr_f6589fadc70243b082688b2b2946aec0/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}

See something wrong? Report a mistake