A Multimodal Entity Linking Dataset with Diversified Contextual Topics and Entity Types
Lists
Only you see this
Abstract
Multimodal Entity Linking (MEL) which aims at linking mentions with multimodal contexts to the referent entities from a knowledge base (e.g., Wikipedia), is an essential task for many multimodal applications. Although much at- tention has been paid to MEL, the shortcom- ings of existing MEL datasets including lim- ited contextual topics and entity types, simpli- fied mention ambiguity, and restricted avail- ability, have caused great obstacles to the re- search and application of MEL. In this pa- per, we present WIKIDiverse, a high-quality human-annotated MEL dataset with diversi- fied contextual topics and entity types from Wikinews, which uses Wikipedia as the cor- responding knowledge base. A well-tailored annotation procedure is adopted to ensure the quality of the dataset. Based on WIKIDiverse, a sequence of well-designed MEL models with intra-modality and inter-modality attentions are implemented, which utilize the visual informa- tion of images more adequately than existing MEL models do. Extensive experimental anal- yses are conducted to investigate the contribu- tions of different modalities in terms of MEL, facilitating the future research on this task. Our dataset and the baseline model are available at https://github.com/wangxw5/wikiDiverse.
BibTeX
@misc{li2023multimodal,
author = {Zhixu Li},
title = {{A Multimodal Entity Linking Dataset with Diversified Contextual Topics and Entity Types}},
year = {2023},
url = {https://openreview.net/forum?id=t6cwi7hzQA},
howpublished = {OpenReview},
}Loading related content...