A Multilingual Bag-of-Entities Model for Zero-Shot Cross-Lingual Text Classification

Sosuke Nishikawa, Ikuya Yamada, Yoshimasa Tsuruoka, Isao Echizen

Abstract

We present a multilingual bag-of-entities model that effectively boosts the performance of zero-shot cross-lingual text classification by extending a multilingual pre-trained language model (e.g., M-BERT). It leverages the multilingual nature of Wikidata: entities in multiple languages representing the same concept are defined with a unique identifier. This enables entities described in multiple languages to be represented using shared embeddings. A model trained on entity features in a resource-rich language can thus be directly applied to other languages. Our experimental results on cross-lingual topic classification (using the MLDoc and TED-CLDC datasets) and entity typing (using the SHINRA2020-ML dataset) show that the proposed model consistently outperforms state-of-the-art models.
BibTeX
@inproceedings{nishikawa2022multilingual,
  author = {Sosuke Nishikawa and Ikuya Yamada and Yoshimasa Tsuruoka and Isao Echizen},
  title = {{A Multilingual Bag-of-Entities Model for Zero-Shot Cross-Lingual Text Classification}},
  booktitle = {CoNLL},
  year = {2022},
  doi = {10.18653/v1/2022.conll-1.1},
}
Figure