TL;DR
This paper proposes a novel probabilistic method for resolving structural ambiguities in language, like prepositional phrase attachment, by leveraging semantic hierarchies (WordNet). Instead of directly estimating probabilities for individual words, they group word senses into classes, allowing for more robust probability estimation even with sparse data.
What Problem Does This Paper Address?
Natural language is rife with ambiguity, meaning a single sentence can have multiple valid interpretations. A common example is prepositional phrase (PP) attachment, where a phrase like "with a telescope" could modify a verb ("look with a telescope") or a noun ("the man with a telescope"). Resolving this ambiguity is crucial for many Natural Language Processing (NLP) tasks, such as parsing and information extraction. Traditional probabilistic approaches struggle with this due to the "sparse data problem": many word combinations are rare, making it difficult to reliably estimate their probabilities. This paper aims to overcome this by using semantic knowledge to generalize word meanings.
Methods
The core of the paper's contribution lies in its "class-based probabilistic approach." Instead of estimating probabilities for individual word senses, they use a semantic hierarchy, specifically WordNet's noun taxonomy, to group related senses into classes.
Here's a breakdown of their approach:
- Semantic Hierarchy: They utilize WordNet's "is-a-kind-of" (hypernymy) relation to build a directed acyclic graph of noun senses. Counts from specific word occurrences are "passed up" the hierarchy to their parent concepts.
- Estimating Probabilities: To estimate the probability of a word sense appearing in a particular argument slot of a verb (e.g., "chicken" as the object of "eat"), they don't use the specific sense of "chicken" directly. Instead, they find a suitable hypernym (like "food") that has enough associated data to estimate probabilities reliably.
- Finding Similarity-Classes: A key innovation is their "similarity-class" technique. They identify a hypernym (a "similarity-class") for a given word sense and verb/argument position such that the probabilities of the verb/argument appearing with the senses within that class are similar. This is achieved by using a chi-squared test on contingency tables of counts at the children of a hypernym. The "top" of this similarity class is the chosen level of generalization.
- PP-Attachment: For PP-attachment, they compare probabilities of the prepositional phrase attaching to the verb versus the noun. They use their class-based approach to estimate these probabilities, selecting the sense of the noun that maximizes the relevant probability.
Datasets
- Training Data: The primary training data is derived from the Penn Treebank, processed into "co-occurrence triples" of (verb lemma, argument position, noun lemma). They also extracted triples from unambiguous attachment cases to augment the training set.
- Test Data: The test data is also from the Penn Treebank, consisting of 4-tuples (verb, noun1, preposition, noun2) with their correct attachment site.
- WordNet: WordNet (version 1.6) is used as the semantic hierarchy for noun senses.
Key Insights and Ideas
- Generalization through Semantics: Grouping word senses into semantic classes is a powerful way to combat the sparse data problem in NLP. This allows for more robust probability estimates by leveraging information from related words.
- Adaptive Generalization Level: The "similarity-class" technique is crucial. It dynamically determines the appropriate level of generalization in the semantic hierarchy for each specific word, verb, and argument position, rather than relying on a fixed, arbitrary level.
- WordNet's Utility: WordNet, with its rich hypernymy structure, proves to be a valuable resource for building these semantic classes and improving NLP model performance.
- PP-Attachment as a Testbed: The PP-attachment problem is a good benchmark for structural disambiguation due to its prevalence and the availability of standard datasets.
Key Results
- The paper demonstrates the effectiveness of their class-based approach on the PP-attachment task.
- Their "similarity-class" technique generally outperforms fixed generalization levels (e.g., always selecting the root of the hierarchy).
- In cases where the WordNet hierarchy is well-populated for a given word, their method achieves high accuracy.
- When WordNet is sparsely populated, their method gracefully defaults to a simpler approach (similar to Hindle and Rooth's) by selecting the "root" of the hierarchy.
- While their overall accuracy on the 3097 test cases (80.3%) is below the state-of-the-art at the time, it is comparable to other similar approaches and shows improvement over simpler methods.
Limitations and Future Work
- Sparsity in WordNet: The performance of their method is heavily dependent on how well WordNet is populated for the specific word senses encountered. When WordNet is sparse, their technique defaults to a less sophisticated method.
- Data Noise: The authors acknowledge that the noise introduced by not having sense-disambiguated data can dissipate as counts are passed up the hierarchy, but it remains a factor.
- Future Work: The authors suggest integrating these probabilities into dependency structure models for parse selection. They also propose incorporating additional knowledge, such as subcategorization information, to further improve disambiguation. They also mention that the significance level (alpha) for their statistical tests could be treated as a parameter to be optimized.
Summary updated