Abstract
Visual Question Answering (VQA) systems target answering open-ended textual questions given input images. Recent research has shown that state-of-the-art models tend to produce answers exploiting biases and shortcuts in the training data instead of performing the required reasoning steps. We present VisQA, a visual analytics tool that exposes the key element of state-of-the-art neural models—attention maps in transformers. Our working hypothesis is that reasoning steps leading to model predictions are observable from attention distributions. The design of VisQA was motivated by well-known bias examples and evaluated by experts, providing evidence that it makes the inner workings of models accessible to users.
The Challenge of Opaque Reasoning
Visual Question Answering (VQA) is a complex task where a model must answer a text-based question about an image (e.g., "Is the person wearing a red hat?"). While state-of-the-art Transformer models perform well, they are notorious for "cheating." Instead of looking at the image, they often rely on statistical biases in the training data—for instance, answering "yellow" to any question about a banana without actually verifying its color.
The challenge is that Transformers are incredibly opaque. With hundreds of "attention heads" processing information in parallel, it is nearly impossible for researchers to pinpoint why a model failed or where it is taking shortcuts. VisQA aims to provide the "X-ray" needed to diagnose these internal reasoning failures.
The VisQA Interface and Methodology
The authors introduce VisQA, an interactive web-based tool that exposes the inner workings of Vision-Language (VL) Transformers (specifically variants of LXMERT).
Key technical features include:
- Attention Map Visualization: It renders the pairwise interactions between words and image objects as heatmaps, showing exactly what the model is "looking at" for every word in the question.
- The "k-number" Metric: To manage the complexity of hundreds of attention maps, the authors use a "k-number" to quantify how "peaky" or "spread out" an attention head is. A low k-number means the head is laser-focused on one specific item.
- Interactive Pruning: Users can manually "turn off" specific attention heads in real-time to see how the model’s final answer changes, helping to identify which heads are responsible for errors.
Understanding the VL-Transformer Architecture
VisQA is designed to analyze models that separate processing into distinct streams before fusing them. As shown in the architecture diagram below, the model processes language and vision independently before cross-modal layers allow the modalities to interact.
Key Insights and Results
- The Vision Bottleneck: Reasoning failures are often caused by the "noisy" object detector missing an object entirely. When the model can't find the "knife" it was asked about, it defaults to a biased guess based on surrounding context.
- Oracle Transfer: A novel training strategy where a model is first trained on "perfect" ground-truth object data (the Oracle). These weights are then transferred to a model using "noisy" real-world object detectors, helping the real-world model adopt cleaner reasoning patterns.
- Pruning for Performance: One of the most surprising findings was that disabling certain attention heads actually improved accuracy on logical questions (using the word "and") by 10%. This suggests that some heads act as "distractors" that confuse the model's logic.
- Expert Intuition: In a user study, deep learning experts using VisQA were able to correctly identify whether a model was reasoning or using bias with 75% precision, proving that visualization is a viable path to model interpretability.
Conclusion
VisQA demonstrates that visualization is a viable path to model interpretability in the complex domain of vision-language reasoning. By exposing attention distributions and providing tools for interactive manipulation, it allows researchers to move beyond simple accuracy metrics and understand the "why" behind model decisions. Future work will focus on scaling these visualizations to even larger transformer architectures and pure vision transformers (ViT).
Summary updated