Abstract
This paper explores the large-scale acquisition of sense-tagged examples for Word Sense Disambiguation (WSD). We have applied the “WordNet monosemous relatives” method to construct automatically a web corpus that we have used to train disambiguation systems. The corpus-building process has highlighted important factors, such as the distribution of senses (bias). The corpus has been used to train WSD algorithms that include supervised methods (combining automatic and manually-tagged examples), minimally supervised (requiring sense bias information from hand-tagged corpora), and fully unsupervised. These methods were tested on the Senseval-2 lexical sample test set, and compared successfully to other systems with minimum or no supervision.
Introduction
Word Sense Disambiguation (WSD)—the task of determining the correct meaning of a word in context—is a fundamental challenge in Natural Language Processing. While supervised learning models achieve high performance, they are constrained by the "knowledge acquisition bottleneck": the high cost and slow pace of producing hand-tagged training data.
To address this, we propose a method to automatically harvest training examples from the web using the monosemous relatives method. By identifying words in WordNet that have only one meaning (e.g., "cathedral" as a relative for the building sense of "church"), we can query search engines for these relatives, collect the surrounding text, and substitute the relative with the target ambiguous word to create synthetic, sense-tagged training data.
The Role of Bias
A central finding of this research is that the distribution of senses within the training set—referred to as bias—is the single most critical factor in the effectiveness of automatically retrieved data. If the training distribution does not align with the real-world distribution (the test set), performance suffers significantly.
As shown in the data, moving from a "no bias" (uniform) distribution to one that approximates the target domain (like Semcor or Senseval-2 bias) drastically improves the baseline performance (1st sense). Interestingly, while the training examples always improve upon the 1st sense baseline, the magnitude of that improvement is largest when the bias is unknown or uniform, suggesting that the training data provides the most value when the system cannot rely solely on frequency heuristics.
Experimental Results
We evaluated our web-trained systems against the Senseval-2 lexical sample benchmark. Our models utilized Decision Lists to process both local features (immediate word context) and topical features (broader keywords).
We tested two primary configurations:
- Minimally Supervised: This model uses the sense distribution (bias) from a small hand-tagged sample (Semcor) but relies on the web corpus for actual training text.
- Fully Unsupervised: This model uses an automatic ranking system to estimate bias without any hand-tagged input.
Our results demonstrate that the web-based corpus is highly effective. The minimally supervised system achieved 49.8% recall, outperforming established systems like UNED. Even the fully unsupervised version reached 43.3% recall, which significantly exceeded contemporary unsupervised competitors that typically performed in the 26-36% range.
Conclusion and Limitations
While automatically retrieved web data can successfully bypass the manual tagging bottleneck, several challenges remain. Google snippets are often noisy or incomplete, which can hinder the extraction of local features. Furthermore, the "substitution problem"—where a specific relative like "Protestant Church" is replaced by the more general "church"—can introduce semantic drift.
Ultimately, this work highlights that the success of WSD systems depends less on the sheer volume of data and more on how well the training distribution matches the target domain. Future work will focus on more sophisticated filtering of web examples and exploring advanced machine learning architectures to better handle noisy synthetic data.
Summary updated