Automatic clustering of collocation for detecting practical sense boundary

Saim Shin, Key-Sun ChoiACL 2004

TL;DR

This paper tackles the challenge of ambiguous word senses in dictionaries by proposing a novel method to automatically discover practical sense boundaries using word collocations from large corpora. By clustering these collocations, the authors demonstrate a way to align dictionary senses with how words are actually used, offering a more robust approach for natural language processing.

What Problem Does This Paper Address?

Dictionaries and thesauri often struggle with defining clear boundaries for word senses, especially for homonymous words (words with the same spelling but different meanings). This ambiguity creates a bottleneck for natural language processing (NLP) systems that rely on accurate word sense disambiguation (WSD). Existing dictionaries may contain senses that are rarely, if ever, used in real-world text (impractical senses), or they might fail to capture nuances of meaning that emerge from corpus usage. This paper aims to bridge this gap by automatically identifying "practical senses" directly from large text corpora, thereby revising and improving the sense boundaries found in traditional dictionaries.

Methods

The core of this paper's methodology lies in leveraging collocations – words that frequently appear together – to infer word senses. The process involves several key steps:

  1. Collocation Extraction: For a given "central word," the system extracts surrounding words within a defined window size from a large corpus.
  2. Normalization: This step aims to clean the extracted collocations.
  3. Statistical Filtering: Words with high frequency that appear regardless of semantic features (likely noise) are removed using tf-idf values.
  4. Latent Semantic Indexing (LSI): LSI is applied to reduce the dimensionality of context vectors and capture hidden semantic features, effectively refining the representation of collocations.
  5. Clustering: The normalized collocations are then clustered. The assumption is that contextual words used in the same sense of a central word will exhibit similar collocation patterns. The paper explores various clustering algorithms, including:
  6. K-means clustering

    • Buckshot
    • Committee Based Clustering (CBC)
    • Markov Clustering (M1, M2)
    • Fuzzy clustering (F1, F2) The similarity measure used is cosine similarity between sense vectors.
  7. Sense Boundary Determination: After clustering, the results from multiple algorithms are combined. A voting mechanism is used to determine the final number of senses for a central word, based on the consensus of the clustering algorithms. The paper also describes a process for mapping clusters across different methods to identify equivalent senses.

Datasets

  • English Corpus: Penn Treebank corpus.
  • Korean Corpus: KAIST corpus.
  • English Dictionaries/Benchmarks:
    • WordNet 1.7 (Fine-grained - WF, Coarse-grained - WC)
    • SENSEVAL English lexical samples (for evaluation of agreement).
  • Korean Dictionaries:
    • Korean Unabridged Dictionary (KD) (fine-grained)
    • Yonsei Dictionary (YD) (corpus-based)

Key Insights and Ideas

  • Collocations as Sense Indicators: The fundamental idea that the patterns of words appearing alongside an ambiguous word can reveal its specific meaning is a powerful insight. This moves beyond simple word co-occurrence to analyzing structured contextual relationships.
  • Corpus-Driven Sense Discovery: The paper champions the idea that practical word senses are best discovered from actual language usage (corpora) rather than solely relying on expert-defined dictionary entries, which can become outdated or incomplete.
  • Ensemble of Clustering Methods: Using multiple clustering algorithms and combining their results (via voting) is a robust strategy to mitigate the weaknesses of any single algorithm and arrive at a more stable sense boundary.
  • Bridging Dictionary and Corpus Senses: The work highlights the discrepancy between traditional dictionary senses and corpus-based senses, suggesting that automatic methods are crucial for aligning these two.

Key Results

  • The proposed method demonstrated similar results to corpus-based dictionaries and sense-tagged corpora in experiments.
  • The clustering results were found to be comparable to the coarse-grained senses in WordNet (WC), suggesting the method effectively captures practical usage.
  • In terms of average agreement for cluster elements, K-means and Buckshot algorithms showed higher agreement, with random initial decisions in K-means and fuzzy clustering leading to higher agreement but slower hierarchical decisions.
  • The paper successfully identified practical sense boundaries for words like "chair," distinguishing between senses like "professor" and "furniture."

Limitations and Future Work

  • Limitations:
    • The paper acknowledges that the experimental results differ from fine-grained, manually analyzed sense distributions (likely expert-curated dictionary senses).
  • The context features used for Korean were limited to 1,000 nouns based on their frequency.
  • Future Work:
    • The authors propose combining their results with practical thesauri automatically.
  • The method could be applied to evaluate and tune existing senses in dictionaries.
  • The ultimate goal is to develop an automatic mechanism for adjusting and constructing knowledge bases like thesauri that are practical and rich in corpus-derived knowledge.

Summary updated

BibTeX
@inproceedings{shin2004automatic,
  author = {Saim Shin and Key-Sun Choi},
  title = {{Automatic clustering of collocation for detecting practical sense boundary}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2004},
}
Figure