Subject-Dependent Co-Occurrence and Word Sense Disambiguation

Joe A. Guthriee, Louise Guthrie, Homa Aidinejad, Yorick WilksACL 1991

TL;DR

This paper introduces "subject-dependent neighborhoods" derived from dictionary definitions, which capture word associations specific to a particular domain. These neighborhoods are then used to significantly improve word sense disambiguation (WSD) by providing more contextually relevant word associations than traditional, general-purpose methods.

What Problem Does This Paper Address?

Understanding the meaning of a word often depends on its context. For example, the word "bank" can refer to a financial institution or the side of a river. This ambiguity, known as word sense disambiguation (WSD), is a fundamental challenge in natural language processing (NLP). Traditional WSD methods often rely on word associations gathered from large, general corpora. However, these general associations can be too broad and may not accurately reflect the specific meaning of a word in a particular domain (e.g., "bank" in a financial context versus a geographical one). The authors aim to address this by creating word association sets that are tailored to specific subject areas, thereby improving the accuracy of WSD.

Methods

The core innovation is the creation of subject-dependent co-occurrence neighborhoods. The authors leverage the Longman Dictionary of Contemporary English (LDOCE), which annotates many word senses with subject codes (e.g., "Economics," "Business," "Automotive").

  1. Subject-Specific Definition Extraction: For a given subject code, they collect all sense definitions from LDOCE that share that code.
  2. Co-occurrence Matrix Construction: They then build a co-occurrence matrix for the 2,187 words in LDOCE's defining vocabulary. The matrix entry fxy represents how many times word x co-occurred with word y within the collected subject-specific definitions.
  3. Relatedness Measure: A relatedness statistic r(x,y) = fxy / (fx * fy) is used to quantify how strongly words are associated.
  4. Neighborhood Formation: A "neighborhood" for a word is formed by selecting the k (e.g., 10) words with the highest relatedness statistic to that word within the subject-specific context.
  5. Word Sense Disambiguation (WSD): To disambiguate a word in a given text, the subject-dependent neighborhoods are intersected with the text. The subject code whose neighborhood shows the greatest overlap with the text is selected. If initial overlaps are too small, an iterative process expands the neighborhoods by incorporating more strongly associated words.

Datasets

  • Longman Dictionary of Contemporary English (LDOCE): Used as the primary source for definitions, word senses, and subject code annotations. The defining vocabulary consists of 2,187 words.
  • Collins COBUILD English Language Dictionary: Used for example sentences to test the WSD method.

Key Insights and Ideas

  • Context is King (Even for Word Associations): The paper strongly supports the idea that word associations are not universal but are highly dependent on the subject domain. A word's "company" truly matters, and where that company is found (i.e., the domain) is crucial.
  • Dictionary Definitions as a Corpus: LDOCE's defining vocabulary, despite its controlled nature and relatively small size (compared to large web corpora), proves to be a valuable resource for generating subject-specific co-occurrence data due to its rich sense distinctions and subject tagging.
  • "Null Code" for Generic Senses: The concept of a "null code" neighborhood is insightful, as it captures more generic or common associations that might not fit neatly into a specific subject category, potentially revealing underlying semantic connections.
  • Hierarchical Subject Codes: The acknowledgment of a hierarchical structure for subject codes (even if not fully implemented in this paper) suggests a path for more nuanced domain modeling.

Key Results

  • Improved Disambiguation: The subject-dependent neighborhoods demonstrably help in distinguishing between word senses that fall into different subject areas. For example, the "bank" neighborhoods for "Economics" and "Business" show distinct sets of associated words, reflecting the theoretical vs. practical nature of these domains.
  • Intersection Analysis: The paper shows that the intersection of subject-dependent neighborhoods with the text is a powerful indicator of the correct sense. For instance, in Example 1 ("The aircraft turned, banking slightly."), only the "Automotive" (AU) neighborhood had significant overlap with the sentence.
  • Qualitative Differences: Tables 1-10 clearly illustrate how the same word ("metal," "bank") yields different sets of associated words depending on the subject code (Economics, Business, Engineering, Automotive, Gambling, Medicine).

Limitations and Future Work

  • Limited Textual Data: The authors acknowledge that the amount of text available within LDOCE for any single subject area is limited. This is a constraint on the richness and representativeness of the generated neighborhoods.
  • Controlled Vocabulary: LDOCE uses a controlled defining vocabulary, which might not fully represent the diversity of language used in real-world texts.
  • Sample Sentences: The sample sentences in LDOCE were constructed with non-native English speakers in mind, which could introduce biases.
  • Future Work:
    • Larger Corpora: The authors plan to use actual documents from specific subject areas (large corpora) to obtain neighborhoods that more accurately reflect real-world texts.
  • Hierarchical Subject Codes: Further exploration of the restructured subject code hierarchy (Slator, 1988) is suggested for more refined domain modeling.
  • Applications: Beyond WSD, subject-dependent neighborhoods are being explored for information retrieval (query expansion) and text subject area determination.
  • Experimental Testing: Extensive experiments are underway to test the method's effectiveness on large numbers of sentences and compare it with other WSD techniques.

Summary updated

BibTeX
@inproceedings{guthriee1991subject,
  author = {Joe A. Guthriee and Louise Guthrie and Homa Aidinejad and Yorick Wilks},
  title = {{Subject-Dependent Co-Occurrence and Word Sense Disambiguation}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {1991},
}
Figure