TL;DR
This paper introduces ALOHa, a novel metric for detecting object hallucinations in image captions that leverages Large Language Models (LLMs) for greater flexibility and accuracy. ALOHa significantly outperforms existing methods like CHAIR, especially on datasets with objects outside the standard MS COCO categories, by providing a more robust and generalizable measure of hallucination.
What Problem Does This Paper Address?
Despite impressive advancements in visual description models, a persistent issue is "object hallucination" – when generated captions mention objects that are not actually present in the image. Existing metrics, like CHAIR, are limited because they rely on a fixed vocabulary of objects (primarily from MS COCO) and struggle with novel or out-of-domain categories. This makes it difficult to accurately evaluate and improve captioning models in real-world scenarios where a wider range of objects and concepts are encountered. The paper aims to address this by developing a more flexible, localized, and generalizable metric for detecting these hallucinations.
Methods
ALOHa is a three-stage process:
- Object Extraction: It uses an LLM (specifically, a prompt-engineered approach with ChatGPT) to extract visually grounded objects from the candidate caption, reference captions, and image detections. This is a key departure from CHAIR, which uses fixed string matching. The LLM is instructed to include attributes, avoid non-visual elements (like sound or emotion), and mark uncertain objects with "(possibly)".
- Object Filtering and Semantic Representation: Candidate and reference object sets are refined. For instance, uncertain objects in the candidate caption are removed to avoid unfair penalties, while attributes are preserved. Semantic similarity is then computed for each object pair using S-BERT embeddings.
- Object Matching: A maximum-similarity linear assignment (using the Hungarian algorithm) is performed between the candidate and reference object sets. The resulting scores, called ALOHa scores, indicate the degree of hallucination for each object. The minimum ALOHa score across all objects in a caption is then used as the overall caption-level hallucination score.
Datasets
- HAT (HAllucination Test): A newly introduced, gold-standard dataset for hallucination detection. It consists of 490 samples (90 validation, 400 test) with expert annotations for both word-level and caption-level hallucinations.
- nocaps-FOIL: A dataset constructed by the authors based on the nocaps dataset. It extends the FOIL concept to objects outside the MS COCO categories, including in-domain, near-domain, and out-of-domain splits to test generalizability.
- Flickr30k: Used for qualitative examples and evaluation.
- MS COCO: The basis for the original FOIL dataset and a source for object detections.
Key Insights and Ideas
- LLMs for Open-Vocabulary Parsing: The core innovation is leveraging LLMs for object extraction, enabling ALOHa to handle a much wider range of objects and concepts than fixed-vocabulary methods like CHAIR. This makes the metric significantly more generalizable.
- Semantic Similarity over Exact Matching: By using semantic embeddings (S-BERT) and a Hungarian matching algorithm, ALOHa can capture nuanced relationships between objects, allowing it to penalize "near misses" (e.g., "wolf" for "dog") more appropriately than simple string matching.
- Handling Uncertainty: The prompt engineering for the LLM explicitly addresses uncertain language (e.g., "possibly," "or") in captions, preventing unfair penalties for models that express caution.
- Localized Hallucination Scores: ALOHa provides object-level hallucination scores, which is crucial for understanding where in the caption the hallucination occurs, unlike metrics that only provide a binary caption-level decision.
Key Results
- Improved Performance on HAT: ALOHa correctly identifies 13.6% more hallucinated objects than CHAIR on the HAT dataset.
- Superior Generalizability: ALOHa outperforms CHAIR by 30.8% on the nocaps-FOIL dataset, demonstrating its effectiveness with objects beyond the MS COCO categories.
- Localization Accuracy: ALOHa achieves significantly higher Localization Accuracy (LA) than CHAIR on HAT, indicating its ability to pinpoint hallucinated objects.
- Robustness to Detection Quality: Ablation studies show that ALOHa is robust to variations in the quality of object detections.
- LLM Choice Matters: While ChatGPT (GPT-3.5) provides the best balance of precision and error rate, other LLMs can also be used, though with some performance degradation.
Limitations and Future Work
- Non-determinism of LLMs: The use of LLMs introduces potential non-determinism and raises concerns about long-term reproducibility, as models can change or become unavailable.
- Reference Caption Dependency: ALOHa, like CHAIR, still requires reference captions, which can be a limitation if they are impoverished or unavailable. The authors highlight reference-free localized hallucination detection as an important future research area.
- Computational Costs: LLMs incur higher computational, monetary, and environmental costs compared to simpler metrics like CHAIR.
- Bias: The authors acknowledge that the HAT dataset, being a subset of MS COCO, may inherit biases. Future work aims to address this by expanding HAT with out-of-domain data and better bias control.
- Future Exploration: Detecting complex hallucinations in actions, quantities, and abstract concepts remains an exciting area for future work. The authors also suggest exploring LLMs specifically trained for semantic similarity between words.
Summary updated