Introduction: The Overconfidence Problem in VQA
Deep learning models often exhibit overconfidence, providing high-certainty answers to nonsensical or irrelevant inputs. This is particularly problematic in Visual Question Answering (VQA), where "anomalies" can manifest in multiple ways: out-of-distribution (OOD) images, OOD questions, or irrelevant image-question pairs.
Standard anomaly detection techniques, such as Maximum Softmax Probability (MSP), often fail in VQA. The authors identify a "fusion masking" effect where the multimodal fusion layer "smoothes over" input anomalies, leading the model to produce a sharp, confident output based on language bias alone, even if the visual input is irrelevant or nonsensical.
Methodology: Shifting Focus to Attention
To address this, the researchers propose moving the detection mechanism from the final output to the internal attention modules.
The framework introduces two key innovations:
- Maximum Attention Probability (MAP): Instead of output confidence, the model evaluates its attention weights. A low MAP score indicates that the model cannot find a strong correlation between visual regions and question tokens, serving as a "reasoning-based" indicator of confusion.
- Maximum Entropy Regularization: Models are fine-tuned with a regularization term that explicitly trains them to produce a uniform (high entropy) attention distribution when encountering anomalous data. This "look everywhere and nowhere" behavior makes anomalies significantly easier to distinguish via the MAP score.
These methods are model-agnostic and can be integrated into various architectures like BUTD, BAN, or MCAN without structural changes.
Experimental Evaluation
The study tested the proposed RA-MAP (Regularized Attention - Maximum Attention Probability) method across diverse datasets, including VQA v2 for in-distribution data, and MNIST, CIFAR-10, and IMDB reviews for various OOD tasks.
Key Results
RA-MAP demonstrated substantial improvements in anomaly detection performance:
- High Precision: AUROC scores for detecting OOD images or questions reached 95-100%, compared to the 50-60% achieved by standard MSP.
- Task Accuracy: Unlike previous "Outlier Exposure" methods that degrade performance on normal tasks, attention regularization maintains high VQA accuracy (e.g., a negligible 0.7% drop for the BUTD model).
- Versatility: The benefits were consistent across four major VQA architectures.
Conclusion and Future Directions
By treating attention as a proxy for reasoning, this work provides a robust way to identify "unanswerable" inputs. While the method currently requires some proxy anomaly data for fine-tuning, it represents a significant step toward more reliable and trustworthy VQA systems. Future work may explore granular detection using individual heads in multi-head attention systems and real-world user studies for assistive technologies.
Summary updated