A Multimodal Entity Linking Dataset with Diversified Contextual Topics and Entity Types

Zhixu LiOpenReview 2023 · OpenReview

Abstract

Multimodal Entity Linking (MEL) which aims at linking mentions with multimodal contexts to the referent entities from a knowledge base (e.g., Wikipedia), is an essential task for many multimodal applications. Although much at- tention has been paid to MEL, the shortcom- ings of existing MEL datasets including lim- ited contextual topics and entity types, simpli- fied mention ambiguity, and restricted avail- ability, have caused great obstacles to the re- search and application of MEL. In this pa- per, we present WIKIDiverse, a high-quality human-annotated MEL dataset with diversi- fied contextual topics and entity types from Wikinews, which uses Wikipedia as the cor- responding knowledge base. A well-tailored annotation procedure is adopted to ensure the quality of the dataset. Based on WIKIDiverse, a sequence of well-designed MEL models with intra-modality and inter-modality attentions are implemented, which utilize the visual informa- tion of images more adequately than existing MEL models do. Extensive experimental anal- yses are conducted to investigate the contribu- tions of different modalities in terms of MEL, facilitating the future research on this task. Our dataset and the baseline model are available at https://github.com/wangxw5/wikiDiverse.
BibTeX
@misc{li2023multimodal,
  author = {Zhixu Li},
  title = {{A Multimodal Entity Linking Dataset with Diversified Contextual Topics and Entity Types}},
  year = {2023},
  url = {https://openreview.net/forum?id=t6cwi7hzQA},
  howpublished = {OpenReview},
}

Loading related content...

Figure