TL;DR
This paper tackles the challenge of accurately determining the most common meaning (predominant sense) of a word within a specific domain, which is crucial for improving Word Sense Disambiguation (WSD). The authors demonstrate that automatically acquiring this information from domain-specific text significantly outperforms using general-purpose resources like SemCor, especially for words that are particularly relevant to that domain.
What Problem Does This Paper Address?
Word Sense Disambiguation (WSD) is the task of identifying the correct meaning of a word in a given context. A common and surprisingly effective heuristic in WSD is to simply choose the most frequent sense of a word (the "predominant sense"). This works well because word senses are often highly skewed, meaning one sense is much more common than others. However, the distribution of word senses isn't static; it's heavily influenced by the domain of the text. For example, the word "bank" will likely have a different predominant sense in a financial document than in a geographical one.
The problem is that creating large, manually annotated corpora for every specific domain is prohibitively expensive. Existing resources like SemCor, while valuable, represent a general mix of domains and might not accurately reflect the predominant sense of a word within a specialized field. This paper aims to address this gap by proposing and evaluating a method to automatically acquire domain-specific predominant sense information from raw, untagged text.
Methods
The core of the paper revolves around an automatic method for finding predominant senses, previously proposed by McCarthy et al. (2004). This method works by:
- Building a Thesaurus: It parses raw text, extracts grammatical relations, and creates a thesaurus by identifying a word's "nearest neighbors" based on their co-occurrence patterns.
- Calculating Distributional Similarity: A distributional similarity metric (specifically, Lin's (1998) metric) is used to quantify how similar words are based on their usage patterns.
- Assigning Sense Rankings: For each word, a ranking score is assigned to its senses. This score is derived from the distributional similarity scores of its neighbors, weighted by the semantic similarity between the senses (using the Jiang and Conrath (1997) metric). WordNet (WN) serves as the sense inventory.
- Acquiring Predominant Sense: The sense with the highest ranking score is identified as the predominant sense for that word in that specific corpus.
The novelty lies in applying this automatic method to domain-specific corpora and then evaluating its effectiveness against a general-domain resource (SemCor).
Datasets
The authors created three new, manually sense-annotated gold standard corpora for a sample of English words. These corpora were derived from:
- BNC (British National Corpus): A large, general-domain corpus of written English (approx. 89.7M words). This serves as a balanced corpus.
- FINANCE: A domain-specific corpus extracted from Reuters (approx. 32.5M words), tagged with financial subject codes.
- SPORTS: A domain-specific corpus extracted from Reuters (approx. 9.1M words), tagged with sports subject codes.
These corpora were used for both training the automatic predominant sense acquisition method and for evaluating its performance.
Key Insights and Ideas
- Domain Matters (A Lot): The most striking insight is how significantly word sense distributions and predominant senses can shift between general text and domain-specific text. Even for words not explicitly tied to a domain, their most common meaning can change.
- Automatic Acquisition Outperforms General Resources: The automatic method for acquiring predominant senses from domain-specific text consistently beats using SemCor's predominant sense information, especially when the training data matches the target domain. This suggests that for WSD, domain-specific knowledge is more valuable than general knowledge.
- Salience is Key: The automatic method performs best on words that are "salient" to a particular domain – words whose meanings are particularly relevant or frequently used within that domain.
- Entropy and Skew: Domain-specific corpora tend to exhibit higher entropy (meaning more uncertainty or a less skewed distribution) for word senses compared to general corpora. However, the predominant sense can become more dominant in domain-specific text, leading to lower entropy for that specific sense.
- "Unclear" and "Unlisted" Senses: The creation of the gold standards revealed instances where existing WordNet senses were insufficient or where the word's usage was genuinely ambiguous within the context, highlighting the limitations of even comprehensive sense inventories.
Key Results
- Predominant Sense Accuracy: When training on domain-relevant data, the automatic method achieved significantly higher accuracy in identifying the correct predominant sense compared to using SemCor data. For instance, in the SPORTS domain, the automatic method achieved 43.7% accuracy, while SemCor FS achieved 16.3%.
- Domain-Specific Training Advantage: Using domain-specific training data (e.g., SPORTS training for SPORTS testing) consistently yielded the best results, outperforming training on the general BNC corpus or SemCor.
- Words with Different Predominant Senses: The study found that a substantial proportion of words (31% in SPORTS, 34% in FINANCE) had a different automatically computed predominant sense in the domain-specific corpus compared to the BNC. For these words, using the domain-specific automatic ranking led to much better WSD performance than relying solely on SemCor.
Limitations and Future Work
- Lexical Sample: The study was conducted on a lexical sample of words, not an all-words task. The authors acknowledge that extending these findings to all words would require further work.
- Defining "Substantial Change": The paper proposes using differences in automatic sense rankings between domain-specific and balanced corpora to identify words where domain-specific acquisition is beneficial. However, they note that further work is needed to precisely define and quantify "substantial change."
- Future Directions: The authors plan to:
- Exploit automatic ranking to obtain more nuanced information on sense frequency distributions (beyond just the predominant sense) for specific genres and domains.
- Combine automatic sense acquisition with local context (using collocates in the thesaurus) for more sophisticated contextual WSD.
Summary updated