TL;DR
This paper introduces a novel, almost-unsupervised method for Word Sense Disambiguation (WSD) by automatically acquiring sense examples from a second language (Chinese). By leveraging bilingual dictionaries and large Chinese text corpora, they generate training data that significantly outperforms previous state-of-the-art unsupervised WSD systems on a standard English benchmark.
What Problem Does This Paper Address?
Word Sense Disambiguation (WSD) is the task of identifying the correct meaning of a word in a given context. Traditional supervised WSD methods require large amounts of manually sense-tagged data, which is expensive and time-consuming to create, leading to a "knowledge acquisition bottleneck." While unsupervised methods exist, they often struggle to achieve high accuracy. This paper tackles the challenge of acquiring sufficient, high-quality training data for WSD in an automated and cost-effective manner, aiming to bridge the gap between the need for supervised-level accuracy and the scarcity of labeled data.
Methods
The core of the paper is the automatic acquisition of sense examples using a cross-lingual approach. The process involves:
- Translating English Senses to Chinese: For an ambiguous English word, its senses are looked up in an English-Chinese dictionary.
- Retrieving Chinese Text Snippets: The Chinese translations of each sense are used as queries to search a large Chinese text corpus (e.g., Gigaword Corpus).
- Segmenting and Translating Back to English: The retrieved Chinese text snippets are segmented into words, and then translated back into English word-by-word using a Chinese-English dictionary.
- Forming Sense Examples: The resulting English word lists from the translated snippets serve as "sense examples" for the original English word sense. These examples are treated as bag-of-words, meaning word order is not considered.
The authors also explored two variations:
- Basic System: Uses automatically acquired sense examples directly.
- System with Manual Refinement: Involves a small amount of human effort to select less ambiguous Chinese translations, reducing noise in the acquired sense examples.
- Multiword Detection Module: An additional module to identify and tag multiword expressions, which is integrated with the classifier.
A Naïve Bayes classifier with kernel estimation is used for WSD.
Datasets
- Training Data: Automatically acquired sense examples generated from:
- Large quantities of Chinese text (Gigaword Corpus, Xinhua News Agency, Central News from Taiwan).
- English-Chinese and Chinese-English bilingual dictionaries (Yahoo! Student English-Chinese On-line Dictionary, PowerWord 2002, LDC Mandarin-English Translation Lexicon Version 3.0).
- Evaluation Data:
- Senseval-2 English Lexical Sample dataset: Used for the main evaluation. This dataset contains manually sense-tagged instances for nouns and adjectives. The authors focused on nouns and adjectives due to the complexity of verb senses.
- TWA dataset: A manually sense-tagged corpus for 6 nouns, used for experiments on the impact of training data size.
Key Insights and Ideas
- Cross-lingual mapping as a proxy for sense: The fundamental idea that typologically distant languages often map word senses differently is exploited. A distinct translation in Chinese can serve as a reliable "sense tag" for an English word.
- Automated data acquisition is feasible and effective: The paper demonstrates that it's possible to automatically generate useful training data for WSD, overcoming the bottleneck of manual annotation.
- Noise reduction significantly improves performance: Even a small amount of manual effort to select less ambiguous translations in the second language (Chinese) leads to substantial improvements in WSD accuracy. This highlights the sensitivity of WSD systems to noisy training data.
- More data generally leads to better performance: Experiments show a clear trend where increasing the number of automatically acquired sense examples improves WSD accuracy.
- Multiword handling is crucial: Incorporating a simple multiword detection module further boosts performance, indicating the importance of handling multiword expressions in WSD.
Key Results
- The proposed system ("Sys B" with multiword detection) achieved significantly better accuracy than comparable state-of-the-art unsupervised WSD systems on the Senseval-2 Lexical Sample dataset.
- The system with manually refined translations ("Sys B") outperformed the basic system ("Sys A"), demonstrating the benefit of reducing ambiguity in the cross-lingual mapping.
- The system easily beat the random baseline and also outperformed the Lesk baseline.
- "Sys B" with multiword detection exceeded the performance of UNED, a top unsupervised participant in Senseval-2.
- Experiments on the TWA dataset showed that increasing the number of training sense examples generally leads to statistically significant improvements in WSD accuracy.
Limitations and Future Work
- Ambiguous Chinese Translations: The paper acknowledges that Chinese translations can themselves be ambiguous, potentially leading to noisy sense examples. While manual refinement helped, it was time-consuming.
- Dictionary Coverage: The LDC Mandarin-English lexicon did not cover all Chinese words, leading to discarded information.
- Sense Granularity Mismatch: The Yahoo! dictionary's coarse granularity did not always align with WordNet's fine-grained senses, requiring manual mapping.
- Future Work:
- Investigate using machine translation software instead of bilingual dictionaries for back-translation.
- Explore other language pairs (e.g., Japanese-English).
- Consider using a third language to resolve ambiguities preserved between English and Chinese.
- Explore the reverse process: building English WSD systems using sense examples acquired from Chinese.
- Further investigate automatic methods for segmenting or parsing Chinese corpora to reduce ambiguity before retrieving snippets.
Summary updated