TL;DR
This paper introduces a novel method for generating "pseudosense-annotated datasets" by creating artificial words (pseudosense words) that mimic real ambiguous words. This approach effectively addresses the long-standing challenges of semantic awareness and broad coverage, enabling the creation of large-scale datasets crucial for advancing research in lexical semantics.
What Problem Does This Paper Address?
A significant bottleneck in computational linguistics is the scarcity of manually annotated data, particularly for tasks like Word Sense Disambiguation (WSD). While artificial "pseudosense-annotated datasets" offer a cost-effective solution, existing methods struggle with two key issues:
- Semantic Awareness: Pseudosenses need to reflect actual semantic relationships between senses of a real word, not just random word pairings (homonymy).
- Coverage: The ability to generate a vast number of diverse pseudowords that model a wide range of ambiguous words is limited.
Previous approaches, like the "vicinity-based" method, could achieve good semantic awareness but suffered from poor coverage, especially when requiring a minimum number of occurrences for each pseudosense. This paper aims to overcome these limitations to facilitate large-scale WSD research.
Methods
The paper proposes a similarity-based pseudoword generation approach that leverages the entire WordNet semantic network. Unlike the "vicinity-based" method, which only considers local subgraphs, this new approach uses Personalized PageRank (PPR).
Here's how it works:
- Graph Representation: WordNet is treated as a graph where synsets are nodes and semantic relationships are edges.
- Personalized PageRank (PPR): For each sense of a target ambiguous word, PPR is run. This algorithm computes a probability distribution, essentially measuring the "importance" or semantic similarity of all other synsets in WordNet to the starting sense.
- Pseudosense Selection: The algorithm sorts the synsets based on their PPR scores. It then searches for monosemous (single-sense) words within these ranked synsets. A candidate word is selected as a pseudosense if it meets a minimum occurrence frequency threshold (minFreq) in a given corpus.
- Coverage and Awareness: By exploring the entire WordNet graph, this method significantly expands the search space, addressing the coverage issue. The use of PPR inherently captures semantic similarity, thus promoting semantic awareness.
Datasets
- WordNet 3.0: Used as the primary lexical resource for identifying ambiguous nouns and their senses.
- English Gigaword corpus: Utilized to determine the occurrence frequency of words and to generate the actual pseudosense-annotated sentences.
- Senseval-3 English lexical sample dataset: Used for evaluating the disambiguation difficulty of pseudowords. This dataset contains manually sense-tagged sentences for 20 ambiguous nouns.
The paper also introduces a large dataset of 15,935 pseudowords modeled after ambiguous nouns in WordNet, with the potential to generate at least 1000 annotated sentences per pseudosense.
Key Insights and Ideas
- PPR as a Semantic Bridge: Personalized PageRank proves to be a powerful tool for capturing nuanced semantic similarity across the entire WordNet graph, going beyond local neighborhood analysis.
- Balancing Coverage and Awareness: The similarity-based approach effectively balances the need for broad coverage (modeling many words) with semantic awareness (modeling real semantic relationships).
- MinFreq's Impact: Increasing the minimum frequency requirement (minFreq) for pseudosenses leads to a deeper search within the PPR rankings, potentially relying on more indirect semantic relations, but still yielding good representative pseudosenses on average.
- Pseudowords as Realistic Proxies: The evaluations suggest that these generated pseudowords are strong proxies for real ambiguous words, performing comparably in WSD tasks and preserving sense distinguishability.
Key Results
- Coverage: The similarity-based approach achieves near-complete coverage (over 80% for minFreq=0, and 25% even for minFreq=1000) of ambiguous nouns in WordNet, a significant improvement over the vicinity-based method (which covered only 5 out of 20 nouns in a specific test).
- Disambiguation Difficulty: WSD systems showed significantly better performance (75.14% recall) on similarity-based pseudowords compared to random pseudowords (78.80% recall), and this performance was much closer to the real word performance (73.26% recall). The similarity-based approach also demonstrated a closer recall to real data for 15 out of 20 tested nouns.
- Representativeness: An average representativeness score of 3.12 (on a scale of 1-4) indicates that most pseudosenses are considered good substitutes for their real senses, even when derived from more distant semantic relations.
- Distinguishability: The generated pseudowords effectively preserve the distinguishability of senses, with an overall score of 0.79 (on a scale of 0-1), meaning they don't tend to over-generalize and cover multiple senses.
Limitations and Future Work
- Limitations:
- The paper focuses primarily on nominal pseudowords, leaving extension to other parts of speech for future work.
- While the similarity-based approach significantly improves coverage, the representativeness score can decrease slightly for higher polysemy degrees or when relying on more indirect semantic relations (as indicated by Table 5 and 7).
- Future Work:
- Extend the pseudoword generation technique to other parts of speech (verbs, adjectives, adverbs).
- Utilize the generated pseudosense-tagged dataset for in-depth studies of various WSD paradigms.
Summary updated