TL;DR
This paper introduces a novel unsupervised method, UMFS-WE, to identify the most frequent sense (MFS) of a word using word embeddings. By comparing a word's embedding to embeddings derived from its potential senses (sense embeddings), the approach effectively bypasses the need for expensive sense-annotated corpora, achieving competitive results, especially for Hindi.
What Problem Does This Paper Address?
Word Sense Disambiguation (WSD) is a fundamental task in Natural Language Processing (NLP) that aims to identify the correct meaning of a word in a given context. A common and surprisingly difficult baseline to beat in WSD is the Most Frequent Sense (MFS) heuristic. This heuristic assumes that a word's most common sense is its predominant sense. While straightforward to implement if one has access to large, sense-annotated corpora to determine sense frequencies, creating such corpora is a costly and time-consuming endeavor, especially for languages other than English. This paper tackles the challenge of obtaining MFS information in an unsupervised manner, without relying on any sense-annotated data, thereby democratizing access to this crucial baseline for WSD research across various languages and domains.
Methods
The core of the UMFS-WE approach lies in leveraging word embeddings. The authors first obtain pre-trained word embeddings for the target word. Then, for each potential sense of that word (as defined in WordNet), they construct a "sense embedding." This sense embedding is calculated as the centroid (average) of the word embeddings of words found within the sense's definition, example sentences, and related synsets (like members, glosses, and hypernyms/hyponyms). The MFS is then identified by finding which sense embedding has the highest cosine similarity to the target word's embedding. This method essentially treats sense embeddings as cluster centroids and the word embedding as a point, identifying the closest cluster.
Datasets
- Hindi: An in-house, newspaper sense-tagged dataset of approximately 80,000 polysemous noun entries was used for training and evaluation.
- English:
- SENSEVAL-2 dataset
- SENSEVAL-3 dataset
Key Insights and Ideas
- Word Embeddings as a Proxy for Sense: The paper demonstrates that word embeddings, trained on large unannotated corpora, implicitly capture semantic information that can be used to infer sense frequencies.
- Sense Embeddings as Sense Representatives: By aggregating embeddings of words associated with a specific sense, the authors create meaningful "sense embeddings" that act as representative vectors for each meaning.
- Unsupervised MFS is Achievable and Competitive: The UMFS-WE approach shows that it's possible to achieve good MFS identification without any supervised sense annotations, challenging the notion that MFS is only accessible through costly annotated resources.
- Language and Domain Independence: The methodology is designed to be language and domain independent, requiring only pre-trained word embeddings and a lexical resource like WordNet.
Key Results
- Hindi WSD: UMFS-WE achieved an F-1 score of 62.00% on the Hindi newspaper dataset, outperforming the WordNet First Sense (WFS) baseline (60.49%).
- English WSD:
- On SENSEVAL-2, UMFS-WE achieved an F-1 score of 52.34%, while the SemCor baseline achieved 59.88%.
- On SENSEVAL-3, UMFS-WE achieved an F-1 score of 43.28%, while the SemCor baseline achieved 65.72%.
- Frequency Matters: The paper shows that as the frequency of a noun in the corpus increases, the UMFS-WE approach's performance improves and surpasses the baselines. Conversely, the SemCor baseline's accuracy can decrease for words that appear infrequently in the test corpus but have a defined MFS in SemCor.
Limitations and Future Work
- English Performance Gap: While competitive for Hindi, UMFS-WE did not outperform the SemCor baseline on English datasets, particularly SENSEVAL-3. The authors suggest this might be due to SemCor's specific nature as a highly curated sense-annotated corpus.
- Infrequent Words: The paper notes that improving performance for infrequent words, especially in English, is a future direction.
- Other Parts-of-Speech and Languages: The authors plan to extend their approach to other parts-of-speech and explore its application to more languages beyond Hindi and English.
- Feature Engineering: The paper explores various combinations of WordNet features for sense embedding creation, suggesting that further optimization of these features could yield better results.
Summary updated