Regularizing Attention Networks for Anomaly Detection in Visual Question Answering

Doyup Lee, Yeongjae Cheon, Wook-Shin Han

Abstract

Visual Question Answering (VQA) models are notoriously overconfident, often providing "correct-sounding" answers to nonsensical or irrelevant inputs. This paper introduces a novel anomaly detection framework that uses Maximum Attention Probability (MAP) and a maximum entropy regularization technique to identify five distinct types of VQA anomalies. The proposed method significantly outperforms standard confidence-based methods while maintaining high answer accuracy across multiple state-of-the-art architectures.

Introduction: The Problem of Overconfident VQA

When deploying deep learning models in critical real-world applications—such as VQA systems assisting visually impaired users—reliability is paramount. Ideally, if a user provides a blurry photo or asks a question unrelated to the image, the model should signal uncertainty rather than hallucinating a confident answer.

In unimodal tasks like image classification, researchers typically use Maximum Softmax Probability (MSP) to detect out-of-distribution (OOD) inputs. However, this paper demonstrates that MSP is insufficient for VQA. Due to complex multimodal fusion, input abnormalities often "vanish" during the reasoning process, leading models to produce high-confidence outputs for nonsensical pairings.

VQA models often provide high-confidence answers to nonsensical inputs, such as identifying animals in an MNIST image.

Methods: Monitoring the Confidence of Reasoning

To address this, the authors shift focus from the final output to the internal reasoning process, proposing two key innovations:

  1. Attention-based Anomaly Detection (MAP): Instead of the final answer distribution, the system monitors cross-modal attention maps. In a valid VQA task, the model should "lock on" to specific image regions and question tokens. If the input is anomalous, the attention mechanism struggles to find strong associations. By measuring the Maximum Attention Probability (MAP), the system can flag samples where the model is "confused."
  2. Maximum Entropy Regularization: To sharpen detection, the authors introduce a regularization term during fine-tuning. They force the model to produce a uniform (high entropy) attention distribution when presented with known anomalous data. This trains the model to "spread its gaze" across the entire input when encountering nonsense, making the drop in MAP more pronounced during inference.
The proposed framework monitors attention modules within the VQA pipeline to detect anomalies.

Experimental Results and Insights

The study evaluated robustness across a comprehensive suite of datasets, including OOD images (MNIST, CIFAR-10), OOD questions (IMDB reviews), and trick questions with false premises.

  • Multimodal Vanishing: T-SNE visualizations revealed that while OOD questions are easily identifiable in joint feature space, OOD images often get "lost" after fusion, explaining why MSP fails.
  • Superior Detection Performance: On OOD image tasks, the proposed RA-MAP achieved AUROC scores near 95-100%, compared to the baseline MSP's ~55-60%.
  • Accuracy Preservation: Unlike standard "Outlier Exposure" (OE) which can degrade primary task performance, RA-MAP maintained high VQA accuracy, with only a negligible 0.5% drop in the BUTD model.

Conclusion and Future Work

The proposed MAP and regularization techniques are model-agnostic, showing success across various architectures like BAN and MCAN. While highly effective for most anomalies, the method struggles with "Undefined Answers"—cases where the reasoning is correct but the answer is outside the model's vocabulary. Future work will explore scaling these methods to multi-head attention and more complex real-world distributional shifts.

Summary updated

BibTeX
@inproceedings{lee2021regularizing,
  author = {Doyup Lee and Yeongjae Cheon and Wook-Shin Han},
  title = {{Regularizing Attention Networks for Anomaly Detection in Visual Question Answering}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2021},
  doi = {10.1609/aaai.v35i3.16279},
}
Figure