TL;DR
This paper proposes a novel algorithm to integrate dictionaries and thesauri by using WordNet as a bridge. This approach aims to overcome the limitations of individual resources and create a more comprehensive lexical semantic knowledge base for applications like word sense disambiguation.
What Problem Does This Paper Address?
Many natural language processing (NLP) tasks, such as word sense disambiguation (WSD), machine translation (MT), and information retrieval (IR), rely on rich lexical semantic information. Dictionaries provide detailed definitions but often suffer from inconsistent sense granularity, while thesauri offer structured taxonomies but can be less precise. The challenge lies in reconciling the detailed, sometimes ambiguous, definitions in dictionaries with the broader, more stable classifications in thesauri. This paper tackles this gap by proposing a method to link these two types of resources effectively.
Methods
The core of the paper is an algorithm that maps word senses from a dictionary (specifically, the Longman Dictionary of Contemporary English - LDOCE) to semantic classes in a thesaurus (Roget's Thesaurus) using WordNet as an intermediary. The process involves several steps:
- Extracting Definitions: For a given word, LDOCE sense definitions are retrieved.
- Leveraging WordNet: WordNet synsets (sets of synonyms) related to the word are identified, along with their glosses, hypernyms (parent concepts), and coordinate synsets (sibling concepts).
- Dictionary-WordNet Similarity: A similarity score is computed between each LDOCE definition and each WordNet synset. This score is a weighted sum of overlaps between the definition and the synset's words, its hypernyms, and its gloss.
- Thesaurus-WordNet Similarity: A similarity score is computed between each WordNet synset (considering the synset itself, its hypernyms, and coordinate terms) and each Roget's Thesaurus paragraph (which represents a semantic class). This score is based on the overlap of words between the WordNet synset's components and the Roget's paragraph.
- Mapping: For each LDOCE sense, the algorithm finds the most similar WordNet synset, and then uses that synset to find the most similar Roget's class. This establishes a mapping from the LDOCE sense to a Roget's class.
The algorithm relies on shallow processing and exploits the inherent inter-relatedness of the resources, rather than requiring extensive statistical data.
Datasets
The paper uses the following lexical resources for its tests:
- Longman Dictionary of Contemporary English (LDOCE): Used as the source of dictionary definitions.
- WordNet 1.5 (WN): Used as the mediator between LDOCE and Roget's Thesaurus.
- Roget's Thesaurus (1987 revision): Used as the target for semantic classification.
The paper tested the algorithm on 12 nouns, with varying numbers of senses across these resources.
Key Insights and Ideas
- WordNet as a Crucial Mediator: The paper demonstrates that WordNet's structured hierarchy and synonym sets provide a robust bridge to connect the detailed definitions of dictionaries with the broader semantic categories of thesauri.
- Exploiting Inter-Resource Relationships: The algorithm successfully leverages the overlapping vocabulary and hierarchical structures present across these different lexical resources.
- Balanced Sense Discrimination: The integration approach suggests the potential for achieving a more balanced degree of sense discrimination than any single resource can offer, by combining their strengths.
- Resource Inadequacy: The study highlights that individual resources have limitations, with a significant percentage of senses in LDOCE, WordNet, and especially Roget's, being "unattached" to links in the other resources. This suggests that combining resources is essential for comprehensive coverage.
Key Results
The algorithm's performance was evaluated, categorizing mapping outcomes into "Accurately Mapped," "Incorrectly Mapped," "Unmapped-a," "Unmapped-b," and "Forced Error."
- Overall Accuracy: The "Accurately Mapped" figures across the different stages of mapping (LDOCE to WordNet, WordNet to Roget's, and the combined LDOCE to Roget's) were 68.9%, 75.0%, and 55.4% respectively.
- Error Types: The study found more "false alarms" (Forced Error, Unmapped-b) than "misses" (Incorrectly Mapped, Unmapped-a), suggesting that errors often stem from a lack of targets in the resources rather than fundamental process failures.
- Unattached Senses: A notable finding is the high percentage of unattached senses: 18.9% of LDOCE senses, 11.1% of WordNet synsets, and a substantial 58.1% of Roget's classes had no counterpart in the other resources.
Limitations and Future Work
- Definition Analysis: The paper acknowledges that its definition analysis is shallow, approximating genus terms through weighted sums rather than explicit parsing. It notes that definitions can involve more complex relations (part-of, membership) not handled in this study.
- Future Work: The authors suggest extending the algorithm to a wider variety of words and applying the integrated lexical information to real-world NLP applications to assess its practical value. They also propose exploring definition parsing to potentially improve mapping results.
Summary updated