Regularizing Attention Networks for Anomaly Detection in Visual Question Answering

Doyup Lee, Yeongjae Cheon, Wook-Shin Han

Introduction: The Overconfidence Problem in VQA

Deep learning models often exhibit overconfidence, providing high-certainty answers to nonsensical or irrelevant inputs. This is particularly problematic in Visual Question Answering (VQA), where "anomalies" can manifest in multiple ways: out-of-distribution (OOD) images, OOD questions, or irrelevant image-question pairs.

Examples of multimodal anomalies: an OOD image paired with an in-distribution question (left) and an in-distribution image paired with an OOD question (right).

Standard anomaly detection techniques, such as Maximum Softmax Probability (MSP), often fail in VQA. The authors identify a "fusion masking" effect where the multimodal fusion layer "smoothes over" input anomalies, leading the model to produce a sharp, confident output based on language bias alone, even if the visual input is irrelevant or nonsensical.

Methodology: Shifting Focus to Attention

To address this, the researchers propose moving the detection mechanism from the final output to the internal attention modules.

The standard VQA pipeline highlighting the internal attention and multimodal fusion modules.

The framework introduces two key innovations:

  1. Maximum Attention Probability (MAP): Instead of output confidence, the model evaluates its attention weights. A low MAP score indicates that the model cannot find a strong correlation between visual regions and question tokens, serving as a "reasoning-based" indicator of confusion.
  2. Maximum Entropy Regularization: Models are fine-tuned with a regularization term that explicitly trains them to produce a uniform (high entropy) attention distribution when encountering anomalous data. This "look everywhere and nowhere" behavior makes anomalies significantly easier to distinguish via the MAP score.

These methods are model-agnostic and can be integrated into various architectures like BUTD, BAN, or MCAN without structural changes.

Experimental Evaluation

The study tested the proposed RA-MAP (Regularized Attention - Maximum Attention Probability) method across diverse datasets, including VQA v2 for in-distribution data, and MNIST, CIFAR-10, and IMDB reviews for various OOD tasks.

Key Results

RA-MAP demonstrated substantial improvements in anomaly detection performance:

  • High Precision: AUROC scores for detecting OOD images or questions reached 95-100%, compared to the 50-60% achieved by standard MSP.
  • Task Accuracy: Unlike previous "Outlier Exposure" methods that degrade performance on normal tasks, attention regularization maintains high VQA accuracy (e.g., a negligible 0.7% drop for the BUTD model).
  • Versatility: The benefits were consistent across four major VQA architectures.

Conclusion and Future Directions

By treating attention as a proxy for reasoning, this work provides a robust way to identify "unanswerable" inputs. While the method currently requires some proxy anomaly data for fine-tuning, it represents a significant step toward more reliable and trustworthy VQA systems. Future work may explore granular detection using individual heads in multi-head attention systems and real-world user studies for assistive technologies.

Summary updated

Abstract

For stability and reliability of real-world applications, the robustness of DNNs in unimodal tasks has been evaluated. However, few studies consider abnormal situations that a visual question answering (VQA) model might encounter at test time after deployment in the real-world. In this study, we evaluate the robustness of state-of-the-art VQA models to five different anomalies, including worst-case scenarios, the most frequent scenarios, and the current limitation of VQA models. Different from the results in unimodal tasks, the maximum confidence of answers in VQA models cannot detect anomalous inputs, and post-training of the outputs, such as outlier exposure, is ineffective for VQA models. Thus, we propose an attention-based method, which uses confidence of reasoning between input images and questions and shows much more promising results than the previous methods in unimodal tasks. In addition, we show that a maximum entropy regularization of attention networks can significantly improve the attention-based anomaly detection of the VQA models. Thanks to the simplicity, attention-based anomaly detection and the regularization are model-agnostic methods, which can be used for various cross-modal attentions in the state-of-the-art VQA models. The results imply that cross-modal attention in VQA is important to improve not only VQA accuracy, but also the robustness to various anomalies.
BibTeX
@inproceedings{lee2021regularizing,
  author = {Doyup Lee and Yeongjae Cheon and Wook-Shin Han},
  title = {{Regularizing Attention Networks for Anomaly Detection in Visual Question Answering}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2021},
  doi = {10.1609/aaai.v35i3.16279},
}
Figure