Beyond Seeing: Fine-Grained Preference Validation and Severity-Aware Alignment for Mitigating Visual Hallucinations in Vision-Language Models

Abdulaziz Alqahtani, Alwaleed Alharthi, Muzammil BehzadarXiv 2026 · arXiv

TL;DR

Researchers have developed a sophisticated alignment pipeline to curb visual hallucinations in Vision-Language Models (VLMs) by validating and repairing preference data. By incorporating the severity of errors into the training objective (Direct Preference Optimization), they achieved significant reductions in object hallucinations, though often at the cost of the model becoming more "cautious" and descriptive.

What Problem Does This Paper Address?

Large Vision-Language Models (LVLMs) like LLaVA are remarkably fluent but prone to "hallucinations"—claiming to see objects, attributes, or relationships that simply aren't in the image.

The authors identify a critical bottleneck in fixing this: data quality. Current preference learning (where a model learns to prefer a "good" response over a "bad" one) often relies on noisy datasets. Sometimes the "chosen" response in a pair still contains subtle hallucinations, or the "rejected" response is only slightly worse than the chosen one. Furthermore, standard training treats all errors as equal, failing to distinguish between a minor attribute mistake (e.g., wrong color) and a major existential failure (e.g., inventing a person who isn't there).

Methods

The paper introduces a multi-stage pipeline to refine hallucination-mitigation data:

  1. Normalization: They take fine-grained feedback (categories like object, attribute, or relation) and map them to a three-level severity rubric: Minor (1), Moderate (2), and Major (3).
  2. Validation & Repair: Instead of trusting existing datasets blindly, they use external "judges" (Gemini-2.5-Flash-Lite and GPT-4o-mini) to verify preference pairs. If a "chosen" response is found lacking, the pipeline attempts to repair it using LLaVA-1.5-7B, followed by a re-verification step.
  3. Severity-Aware Alignment: They compare three training strategies using Direct Preference Optimization (DPO):
  4. Standard DPO: Treats all preference pairs equally.
  5. HSA-DPO: Weights the loss based on the severity of the rejected response's errors.
  6. Severity-Margin DPO (Proposed Variant): Introduces a mathematical "margin" into the DPO objective. The more severe the hallucination in the rejected response, the larger the gap the model is forced to maintain between the "good" and "bad" log-probabilities.

Datasets

The study utilizes a collection of 16,143 normalized feedback records and 8,386 preference pairs derived from previous research (specifically building on the HSA-DPO line of work).

For evaluation, they use three primary benchmarks:

  • POPE: Probes object presence via binary (Yes/No) questions.
  • Object HalBench: Uses the CHAIR metric to quantify unsupported object mentions in long-form captions.
  • AMBER: An LLM-free framework that evaluates multiple dimensions of hallucination (Object, Attribute, Relation).

Key Insights and Ideas

  • The "Alignment Tax" of Cautiousness: A recurring theme is that as models become less prone to hallucination, they also become less descriptive. The "AMBER-Cover" metric (visual coverage) decreased across all aligned models, suggesting that models learn to "play it safe" by mentioning fewer details to avoid mistakes.
  • Judges as Gatekeepers: The authors found that zero-shot judging (giving the rubric directly to the LLM judge) was often more effective than two-shot judging for certain metrics. This highlights the difficulty of "calibrating" automated judges for preference data.
  • Severity is a Signal, Not a Silver Bullet: While severity-aware margins helped significantly on Object HalBench, they didn't provide a "uniform superiority" across all benchmarks. Hallucination mitigation remains highly benchmark-dependent.
  • Repairing the "Chosen": The pipeline successfully recovered hundreds of high-quality training pairs by repairing flawed "chosen" responses, proving that even "gold standard" alignment data can benefit from automated cleanup.

Key Results

  • Object HalBench: The "Zero-shot Judged Margin" variant was the clear winner here, slashing the sentence-level hallucination rate (CHAIRs) from 53.00 (base LLaVA) down to 36.00.
  • AMBER: All aligned models showed massive improvements. Hallucination scores (AMBER-Hal) dropped from 35.90 to a range of 25.10–26.90.
  • POPE: Interestingly, the base model retained the highest F1 score (84.18). While alignment increased precision (the model was more often right when it said "Yes"), it decreased recall (it said "No" more often), leading to a slight net drop in F1.

Limitations and Future Work

  • Backbone Constraints: The study focused exclusively on LLaVA-1.5-7B. It is unclear if these severity-aware margins scale identically to much larger or differently architected models.
  • Component Isolation: The authors note that because the "judged" variants changed both the training data (via filtering/repair) and the objective function simultaneously, it is difficult to attribute gains to a single specific factor.
  • Human-in-the-loop: The pipeline relies heavily on LLM judges. The authors suggest that future work should include independent human audits to verify if the "automated preference quality" aligns with human perception of truthfulness.
  • The Coverage Gap: Future research needs to find ways to reduce hallucinations without sacrificing the richness and detail of the model's descriptions.

Summary updated

Abstract

Large vision-language models can produce fluent responses containing objects, attributes, or relations unsupported by the image. We study an implemented alignment pipeline that normalizes released fine-grained hallucination supervision, validates chosen-rejected preference pairs with vision-language judges, repairs rejected chosen responses, and trains LLaVA adapters with standard or severity-aware direct preference optimization. The study uses 16,143 normalized feedback records and 8,386 released preference pairs, comparing direct training with zero-shot and two-shot judged severity-margin variants. The zero-shot variant reduces Object HalBench CHAIRs from 53.00 to 36.00 and CHAIRi from 15.72 to 10.41. Across aligned models, AMBER-Hal decreases from 35.90 to 25.10-26.90, accompanied by reduced visual coverage. In contrast, the base model retains the highest valid POPE adversarial F1, 84.18 versus 83.87-83.91 after alignment. These results support benchmark-dependent reductions in generative hallucination rather than uniform superiority of severity-aware alignment. We distinguish implemented pipeline components from the preference paths used in the reported experiments and discuss the limits of attributing gains to judging, repair, or the training objective individually.
BibTeX
@misc{alqahtani2026beyond,
  author = {Abdulaziz Alqahtani and Alwaleed Alharthi and Muzammil Behzad},
  title = {{Beyond Seeing: Fine-Grained Preference Validation and Severity-Aware Alignment for Mitigating Visual Hallucinations in Vision-Language Models}},
  year = {2026},
  eprint = {2611.20478},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.20478},
}
Figure