Vision Meets Definitions: Unsupervised Visual Word Sense Disambiguation Incorporating Gloss Information
Lists
Only you see this
Abstract
Visual Word Sense Disambiguation (VWSD) is a task to find the image that most accurately depicts the correct sense of the target word for the given context. Previously, image-text matching models often suffered from recognizing polysemous words. This paper introduces an unsupervised VWSD approach that uses gloss information of an external lexical knowledge-base, especially the sense definitions. Specifically, we suggest employing Bayesian inference to incorporate the sense definitions when sense information of the answer is not provided. In addition, to ameliorate the out-of-dictionary (OOD) issue, we propose a context-aware definition generation with GPT-3. Experimental results show that the VWSD performance significantly increased with our Bayesian inference-based approach. In addition, our context-aware definition generation achieved prominent performance improvement in OOD examples exhibiting better performance than the existing definition generation method.
BibTeX
@inproceedings{kwon2023vision,
author = {Sunjae Kwon and Rishabh Garodia and Minhwa Lee and Zhichao Yang and Hong Yu},
title = {{Vision Meets Definitions: Unsupervised Visual Word Sense Disambiguation Incorporating Gloss Information}},
booktitle = {Annual Meeting of the Association for Computational Linguistics},
year = {2023},
url = {https://openreview.net/forum?id=N7XHqSbtyzb},
}Loading related content...