TL;DR
This paper introduces MuLaN, a novel method for automatically generating multilingual sense-annotated datasets for Word Sense Disambiguation (WSD). By leveraging contextualized word embeddings and knowledge bases, MuLaN projects sense labels from high-resource languages (like English) to lower-resource ones, achieving state-of-the-art performance on multilingual WSD tasks.
What Problem Does This Paper Address?
Word Sense Disambiguation (WSD) is a crucial task in Natural Language Processing (NLP) that involves identifying the correct meaning of a word in a given context. However, training effective WSD models, especially for multiple languages, is severely hampered by the lack of large, manually annotated datasets. This "knowledge acquisition bottleneck" is particularly acute for WSD because each word has its own set of senses, making manual annotation extremely time-consuming and expensive. While existing methods have tried to mitigate this through parallel corpora or knowledge bases, they often have limitations, such as requiring parallel data or being restricted to specific word types (like nouns). This paper aims to overcome these limitations by creating high-quality, automatically generated multilingual WSD datasets.
Methods
MuLaN employs a three-step semi-supervised approach:
- Vectorization: It uses a multilingual contextualized word embedding model (Multilingual BERT or m-BERT) to project instances from a labeled source language corpus and text spans from an unlabeled target language corpus into a shared latent space. This allows for cross-lingual representation of word meanings.
- Candidate Production: For each instance in the source corpus, MuLaN identifies potential matching text spans in the target corpus based on cosine similarity in the shared latent space. This step is further refined by using a knowledge base (BabelNet) to filter out candidates that are not valid lexicalizations for the target synset, ensuring semantic coherence.
- Dataset Generation: A backward compatibility test is applied to further refine the candidate lists. Finally, the system generates sense-annotated datasets by selecting the most suitable sentences from the filtered candidates, using a marginalized cosine similarity measure to contextualize the similarity score. This process effectively projects sense labels from a high-resource language to lower-resource ones.
Datasets
- Training Data: The paper uses the concatenation of SemCor and WNG (Princeton WordNet Gloss Corpus) as the labeled source corpus (English). Wikipedia is used as the unlabeled target corpus in various languages.
- Evaluation Data: The paper evaluates its approach on multilingual WSD datasets from SemEval-2013 and SemEval-2015, including both original and revised versions. They also create specific test sets to analyze performance on "Least Common Senses" and "Unseen Senses."
Key Insights and Ideas
- Leveraging Multilingual Embeddings for Data Generation: The core idea is that powerful multilingual embeddings (like m-BERT) can bridge language barriers not just for WSD models, but also for the creation of training data itself.
- Knowledge Base as a Semantic Filter: Using BabelNet to filter candidate word senses is crucial for ensuring the quality of the projected annotations, preventing spurious matches.
- Backward Compatibility Test for Robustness: The backward compatibility test adds an extra layer of validation, ensuring that the projected sense is truly reciprocal, further improving dataset quality.
- Multilingual Training Data Benefits: The experiments demonstrate that training WSD models on datasets generated from multiple languages (MuLaN4L) consistently outperforms training on monolingual data, even for unseen senses. This highlights the power of cross-lingual knowledge transfer.
- Addressing Least Common Senses: MuLaN shows a significant advantage over competitors when focusing on less frequent senses, indicating its ability to generate valuable data for challenging WSD scenarios.
Key Results
- MuLaN-generated datasets, when used to train a reference WSD model, achieve state-of-the-art performance on multilingual WSD tasks, outperforming competitors by several points.
- The MuLaN approach significantly outperforms previous methods, including knowledge-based and other automatic annotation techniques, especially when considering all languages.
- When tested on "Least Common Senses," MuLaN demonstrates a much lower bias towards the most common sense compared to competitors like OneSeC, providing more valuable data for rare senses.
- Training with multilingual data (MuLaN4L) leads to an average improvement of 2 F1 points over monolingual training data, particularly for unseen senses.
Limitations and Future Work
- Limited Sense Coverage: The current approach's sense coverage is limited by the senses present in the source corpus.
- Future Work: The authors plan to extend MuLaN's sense coverage to meanings not included in the source corpus and generalize the approach to transfer labels for other NLP tasks beyond WSD.
Summary updated