Know What You do Not Know: Verbalized Uncertainty Estimation Robustness on Corrupted Images in Vision-Language Models

Mirko Borszukovszki, Ivo Pascal De Jong, Matias Valdenegro-ToroNAACL 2025

TL;DR

This paper investigates how well Visual Language Models (VLMs) can estimate their uncertainty when presented with corrupted images. The study reveals that as image corruption increases, VLMs become less accurate and their confidence estimates degrade, often leading to overconfident wrong answers. This highlights a critical need for more robust uncertainty estimation in VLMs for reliable real-world applications.

What Problem Does This Paper Address?

As AI assistants powered by Large Language Models (LLMs) become more sophisticated, their ability to provide accurate and reliable information is paramount. A key aspect of reliability is understanding a model's uncertainty – how confident it is in its answers. While significant research has focused on uncertainty in text-only LLMs, the integration of visual capabilities into these models (creating VLMs) has introduced new challenges. A major problem is that current VLMs often provide answers with the same high confidence, regardless of whether the answer is correct or a hallucination. This lack of calibrated uncertainty can lead to users over-trusting incorrect information, especially when dealing with imperfect or noisy real-world data. This paper specifically addresses the question of how the severity of image corruption impacts a VLM's ability to accurately estimate its uncertainty.

Methods

The researchers evaluated three state-of-the-art VLMs: GPT-4 Vision, Gemini Pro Vision, and Claude 3 Opus. They employed a "verbalized uncertainty" approach, prompting the models to express their confidence in their answers, often as a percentage or a 95% confidence interval. To test robustness, they introduced various levels of corruption (Gaussian noise, defocus blur, and JPEG compression) to images from three different datasets. The models' accuracy, confidence scores, and refusal rates were meticulously recorded and analyzed. Expected Calibration Error (ECE) was used as a primary metric to quantify the models' calibration.

Datasets

The study utilized three datasets for evaluation:

  1. Easy Visual Question Answering Dataset: A subset of 36 images and questions from the popular VQA dataset (Antol et al., 2015; Goyal et al., 2017), focusing on straightforward image-based questions.
  2. Japanese Uncertain Scenes (JUS) Dataset: A dataset proposed by Groot and Valdenegro-Toro (2024), containing 29 "tricky" questions designed to challenge a model's uncertainty estimation capabilities.
  3. JUS Dataset (Counting Task): A subset of the JUS dataset with 13 challenging counting exercises, where the focus is on uncertainty estimation rather than precise accuracy.

Key Insights and Ideas

  • Corruption is a major challenge for VLM uncertainty: The study unequivocally shows that increasing image corruption significantly degrades VLMs' ability to accurately estimate their confidence.
  • Overconfidence is pervasive: Across all tested models and corruptions, VLMs consistently exhibit overconfidence, meaning their stated confidence is often higher than their actual accuracy. This is particularly pronounced in the counting task.
  • Refusal can be a sign of robustness: Models that refuse to answer when faced with highly corrupted or ambiguous images tend to have better calibration. This suggests that knowing when not to answer is a crucial aspect of reliable AI.
  • GPT-4V shows better robustness: Among the tested models, GPT-4V generally demonstrated superior performance in maintaining calibration and accuracy under corruption, suggesting better underlying robustness.
  • JPEG compression is less disruptive: Compared to Gaussian noise and defocus blur, JPEG compression had a less severe impact on the models' calibration.

Key Results

The study found that as corruption severity increased, the Expected Calibration Error (ECE) generally increased for all models and corruptions, indicating a decline in calibration. For instance, in the "easy" VQA task, GPT-4V achieved the lowest ECE scores across different severity levels and corruptions. In the "hard" VQA task, while all models became more miscalibrated, GPT-4V still performed best. The counting task revealed particularly poor calibration, with models often failing to achieve 95% accuracy for their stated 95% confidence intervals, even when their initial guesses were close. Refusal rates were observed to increase with corruption severity, and higher refusal rates correlated with improved calibration for some models.

Limitations and Future Work

The authors acknowledge several limitations. The calibration plots suffered from underrepresentation of lower confidence bins due to the limited number of data points, especially in the JUS dataset. Manually checking answers was time-consuming, limiting the scale of testing. Future work could explore different prompting strategies (e.g., chain-of-thought), investigate other corruption types, and examine techniques like temperature scaling to potentially improve VLM calibration. The study also suggests that further research is needed to understand why certain corruptions are handled better than others and to develop methods that can detect a wider range of input image corruptions.

Summary updated

BibTeX
@inproceedings{borszukovszki2025know,
  author = {Mirko Borszukovszki and Ivo Pascal De Jong and Matias Valdenegro-Toro},
  title = {{Know What You do Not Know: Verbalized Uncertainty Estimation Robustness on Corrupted Images in Vision-Language Models}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2025},
}
Figure