TL;DR
This paper tackles the "tennis problem" in WordNet, where related concepts like "tennis player" and "racket" are not semantically linked. The authors propose augmenting WordNet with topic-based relationships derived from the CIDE+ dictionary by combining lexical similarity metrics. Their approach significantly improves upon baseline methods, achieving near human-level performance in disambiguating noun senses.
What Problem Does This Paper Address?
Lexical taxonomies like WordNet are invaluable resources for Natural Language Processing (NLP), organizing words into hierarchical structures based on meaning (e.g., "dog" IS_A "canine"). However, WordNet, like many such resources, suffers from a lack of comprehensive semantic coverage. A prime example is the "tennis problem": while WordNet might link "tennis player" to "person," it fails to explicitly connect it to related concepts like "racket," "ball," or "net," which are crucial for understanding the domain. This omission limits the effectiveness of NLP applications in areas like information retrieval and word sense disambiguation. The paper aims to address this by adding "thesaural relations" – topic or domain-based links – to WordNet.
Methods
The core of the paper is a method for disambiguating noun senses from the CIDE+ dictionary's "related word classes" against WordNet synsets. This is achieved by combining three lexical similarity metrics:
- Information Content (IC): Based on Resnik's work, this metric assigns a numerical value to each WordNet synset based on its frequency in a corpus, reflecting how informative the sense is.
- Conceptual Distance (CD): This metric, inspired by Agirre and Rigau, determines the distance between WordNet senses based purely on the taxonomy's structure (depth, density, shortest path), ignoring corpus frequencies.
- Dictionary Overlap (DO): This metric, based on Lesk's approach, measures similarity by counting shared content words in the textual definitions of WordNet synsets and CIDE+ senses.
These individual metrics are then combined using both additive and multiplicative approaches. The paper hypothesizes that multiplicative combinations, acting as filters, will be more effective.
Datasets
- WordNet (version 1.6): The primary lexical taxonomy augmented in this work.
- CIDE+ Machine Readable Dictionary: Used as the source for "related word classes" to provide thesaural information.
- SEMCOR Corpus: A 200,000-word portion of the Brown Corpus, semantically tagged with WordNet senses. This was adapted to evaluate the disambiguation method. The authors note that no existing resource directly maps CIDE+ classes to WordNet senses, so they developed a method to derive suitable test data from SEMCOR by identifying texts where words from CIDE+ categories appeared together, assuming they would be used with senses appropriate to that category.
Key Insights and Ideas
- The "Tennis Problem" is Real and Important: The paper clearly articulates the limitations of purely hierarchical taxonomies and the need for domain-specific, topic-based relationships.
- Combining Metrics is Key: Individual similarity metrics have limitations. Combining them, especially through multiplication, acts as a powerful filtering mechanism, leading to more robust disambiguation.
- Multiplicative Combinations are More Conservative: The paper suggests that when multiple metrics must agree (as in a product), it leads to more reliable disambiguation, as it filters out senses that might be plausible under one metric but not others.
- SEMCOR as a Proxy for Evaluation: The authors ingeniously leverage SEMCOR to create a benchmark for their evaluation, demonstrating a practical way to assess disambiguation methods when direct gold-standard data is scarce.
Key Results
The paper evaluates various combinations of the three metrics. The results show that:
- All implemented metrics perform significantly better than a random baseline.
- The product of all three metrics (DO * IC * CD) achieved the best performance, with an Average Cross Entropy (ACE) score of 5.76.
- This score is remarkably close to the performance of a human annotator (ACE of 5.16), representing a 94% reduction in error rate compared to the random baseline.
- Additive combinations of metrics did not show significant improvement over single metrics, whereas multiplicative combinations performed much better.
Limitations and Future Work
- Limited Scope of CIDE+: The paper focuses on noun synsets and uses a specific thesaurus (CIDE+). The approach might need adaptation for other word classes or different thesauri.
- SEMCOR Adaptation: While ingenious, the method of deriving test data from SEMCOR is a heuristic. The assumption that words appearing together in a text will share the same sense might not always hold perfectly.
- Future Work: The authors suggest that this approach could be extended to other grammatical categories and that the method might be useful for other tasks like clustering senses. The paper also hints at the potential for combining more sophisticated similarity metrics.
Summary updated