VisQA: X-raying Vision and Language Reasoning in Transformers

Theo Jaunet, Corentin Kervadec, Romain Vuillemot, Grigory Antipov, Moez Baccouche, Christian Wolf

Abstract

Visual Question Answering (VQA) systems target answering open-ended textual questions given input images. Recent research has shown that state-of-the-art models tend to produce answers exploiting biases and shortcuts in the training data instead of performing the required reasoning steps. We present VisQA, a visual analytics tool that exposes the key element of state-of-the-art neural models—attention maps in transformers. Our working hypothesis is that reasoning steps leading to model predictions are observable from attention distributions. The design of VisQA was motivated by well-known bias examples and evaluated by experts, providing evidence that it makes the inner workings of models accessible to users.

The Challenge of Opaque Reasoning

Visual Question Answering (VQA) is a complex task where a model must answer a text-based question about an image (e.g., "Is the person wearing a red hat?"). While state-of-the-art Transformer models perform well, they are notorious for "cheating." Instead of looking at the image, they often rely on statistical biases in the training data—for instance, answering "yellow" to any question about a banana without actually verifying its color.

The challenge is that Transformers are incredibly opaque. With hundreds of "attention heads" processing information in parallel, it is nearly impossible for researchers to pinpoint why a model failed or where it is taking shortcuts. VisQA aims to provide the "X-ray" needed to diagnose these internal reasoning failures.

The VisQA Interface and Methodology

The authors introduce VisQA, an interactive web-based tool that exposes the inner workings of Vision-Language (VL) Transformers (specifically variants of LXMERT).

The VisQA interface allows users to explore attention maps, filter by question, and analyze global statistics of attention heads to distinguish reasoning from bias.

Key technical features include:

  • Attention Map Visualization: It renders the pairwise interactions between words and image objects as heatmaps, showing exactly what the model is "looking at" for every word in the question.
  • The "k-number" Metric: To manage the complexity of hundreds of attention maps, the authors use a "k-number" to quantify how "peaky" or "spread out" an attention head is. A low k-number means the head is laser-focused on one specific item.
  • Interactive Pruning: Users can manually "turn off" specific attention heads in real-time to see how the model’s final answer changes, helping to identify which heads are responsible for errors.

Understanding the VL-Transformer Architecture

VisQA is designed to analyze models that separate processing into distinct streams before fusing them. As shown in the architecture diagram below, the model processes language and vision independently before cross-modal layers allow the modalities to interact.

The VL-Transformer architecture consists of language-only and vision-only self-attention blocks, followed by cross-modal attention layers (Vision to Language and Language to Vision).

Key Insights and Results

  • The Vision Bottleneck: Reasoning failures are often caused by the "noisy" object detector missing an object entirely. When the model can't find the "knife" it was asked about, it defaults to a biased guess based on surrounding context.
  • Oracle Transfer: A novel training strategy where a model is first trained on "perfect" ground-truth object data (the Oracle). These weights are then transferred to a model using "noisy" real-world object detectors, helping the real-world model adopt cleaner reasoning patterns.
  • Pruning for Performance: One of the most surprising findings was that disabling certain attention heads actually improved accuracy on logical questions (using the word "and") by 10%. This suggests that some heads act as "distractors" that confuse the model's logic.
  • Expert Intuition: In a user study, deep learning experts using VisQA were able to correctly identify whether a model was reasoning or using bias with 75% precision, proving that visualization is a viable path to model interpretability.

Conclusion

VisQA demonstrates that visualization is a viable path to model interpretability in the complex domain of vision-language reasoning. By exposing attention distributions and providing tools for interactive manipulation, it allows researchers to move beyond simple accuracy metrics and understand the "why" behind model decisions. Future work will focus on scaling these visualizations to even larger transformer architectures and pure vision transformers (ViT).

Summary updated

Abstract

Visual Question Answering systems target answering open-ended textual questions given input images. They are a testbed for learning high-level reasoning with a primary use in HCI, for instance assistance for the visually impaired. Recent research has shown that state-of-the-art models tend to produce answers exploiting biases and shortcuts in the training data, and sometimes do not even look at the input image, instead of performing the required reasoning steps. We present VisQA, a visual analytics tool that explores this question of reasoning vs. bias exploitation. It exposes the key element of state-of-the-art neural models -- attention maps in transformers. Our working hypothesis is that reasoning steps leading to model predictions are observable from attention distributions, which are particularly useful for visualization. The design process of VisQA was motivated by well-known bias examples from the fields of deep learning and vision-language reasoning and evaluated in two ways. First, as a result of a collaboration of three fields, machine learning, vision and language reasoning, and data analytics, the work lead to a better understanding of bias exploitation of neural models for VQA, which eventually resulted in an impact on its design and training through the proposition of a method for the transfer of reasoning patterns from an oracle model. Second, we also report on the design of VisQA, and a goal-oriented evaluation of VisQA targeting the analysis of a model decision process from multiple experts, providing evidence that it makes the inner workings of models accessible to users.
BibTeX
@article{jaunet2022visqa,
  author = {Theo Jaunet and Corentin Kervadec and Romain Vuillemot and Grigory Antipov and Moez Baccouche and Christian Wolf},
  title = {{VisQA: X-raying Vision and Language Reasoning in Transformers}},
  journal = {IEEE Trans. Vis. Comput. Graph.},
  year = {2022},
  url = {https://openreview.net/forum?id=ZzysqGYEKq},
}
Figure