MAGIC-VQA: Multimodal And Grounded Inference with Commonsense Knowledge for Visual Question Answering

Shuo Yang, Siwen Luo, Soyeon Caren Han, Eduard Hovy

Abstract

Visual Question Answering (VQA) requires reasoning across visual and textual modalities, yet Large Vision-Language Models (LVLMs) often lack integrated commonsense knowledge, limiting their robustness in real-world scenarios. To address this, we introduce MAGIC-VQA, a novel framework that enhances VQA by systematically integrating commonsense knowledge with LVLMs. MAGIC-VQA employs a three-stage process: (1) Explicit Knowledge Integration from external sources, (2) By-Type Post-Processing for contextual refinement, and (3) Implicit Knowledge Augmentation using a Graph Neural Network (GNN) for structured reasoning. While GNNs bring greater depth to structured inference, they enable superior relational inference beyond LVLMs. MAGIC-VQA bridges a key gap by unifying commonsensse knowledge with LVLM-driven reasoning, eliminating the need for extensive pre-training or complex prompt tuning. Our framework achieves state-of-the-art performance on benchmark datasets, significantly improving commonsense reasoning in VQA.
BibTeX
@inproceedings{yang2025magic,
  author = {Shuo Yang and Siwen Luo and Soyeon Caren Han and Eduard Hovy},
  title = {{MAGIC-VQA: Multimodal And Grounded Inference with Commonsense Knowledge for Visual Question Answering}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2025},
  url = {https://openreview.net/forum?id=PCkeHy63kC},
}

Loading related content...

Figure