Introduction
Vision-Language Pre-training (VLP) has enabled models like BLIP and VinVL to generate remarkably fluent, human-like captions. However, a troubling paradox has emerged: models that achieve higher scores on standard metrics like CIDEr often exhibit a higher frequency of object hallucination. These models frequently mention objects that are not present in the image—such as describing a "mirror" on a dresser when the visual evidence shows none. While these descriptions are linguistically "plausible," they are unfaithful to the source image, posing significant safety risks for applications in medical imaging and autonomous systems.
Probing the Hallucination Phenomenon
To understand why these models "see" non-existent objects, the researchers conducted a systematic probing study across several dimensions:
- Evaluation Metrics: The study utilized the CHAIR metric to quantify the proportion of hallucinated objects, moving beyond fluency-based metrics that can be easily gamed by language priors.
- Image Encoding: They compared region-based (object detectors), grid-based (CNN), and patch-based (Vision Transformer) encoding strategies to see how visual representation affects faithfulness.
- Objective Decoupling: By isolating Image-Text Contrastive (ITC), Image-Text Matching (ITM), and Image-Conditioned Language Modeling (ICLM) losses, they identified which training objectives drive hallucinations.
The "CIDEr Trap" and Key Insights
A primary finding of the research is the "CIDEr Trap." Industry-standard methods like Self-Critical Sequence Training (SCST), which are designed to maximize CIDEr scores, actually exacerbate hallucinations. By rewarding n-gram overlap, SCST encourages models to include descriptive but incorrect details to boost their scores.
Furthermore, the study revealed that:
- Patch-based features (ViT) outperform region-based features, likely because explicit object detectors can provide insufficient context if they fail to detect an object initially.
- Fine-grained resolution is critical; smaller patch resolutions significantly reduce hallucination rates by providing more precise visual evidence.
- Generative loss (ICLM) is the main culprit behind hallucinations, as it causes the model to rely too heavily on its internal language priors rather than the actual visual input.
Mitigating Hallucination with ObjMLM
To address these issues, the authors introduced Object Masked Language Modeling (ObjMLM). Unlike standard Masked Language Modeling (MLM), which masks random tokens, ObjMLM specifically targets tokens corresponding to objects present in the image. This strategy forces the model to rely strictly on visual cues to reconstruct the masked text, thereby strengthening the alignment between visual objects and textual tokens and reducing the influence of the language model's internal biases.
Results and Future Directions
The implementation of the ObjMLM loss led to a 17.4% reduction in sentence-level hallucination rates ($CHAIR_s$) compared to standard baselines. Despite this progress, modern VLP models still hallucinate in approximately 10% of generated sentences, with performance degrading further on out-of-domain datasets like NoCaps.
Future research will look to expand this probing to attribute and relation hallucinations (such as incorrect colors or spatial interactions), with the ultimate goal of enhancing the controllability and faithfulness of vision-conditioned text generation.
Summary updated