Word Sense Acquisition from Bilingual Comparable Corpora

Hiroyuki KajiNAACL 2003

TL;DR

This paper introduces a novel method for automatically acquiring word senses by leveraging bilingual comparable corpora and dictionaries. By clustering translation equivalents based on their translingual distributional patterns, the approach generates a hierarchy of corpus-relevant word senses, overcoming limitations of manual sense inventory construction.

What Problem Does This Paper Address?

Creating comprehensive inventories of word senses (lexicons) is a crucial but labor-intensive task in Natural Language Processing (NLP). Traditional methods rely on lexicographers' intuition, which is costly, prone to arbitrary divisions of meaning, and often doesn't align with the specific domains of text being analyzed. Existing automated approaches to word sense acquisition often struggle with polysemy (words with multiple meanings) and the need for precise definitions of senses, typically requiring parallel corpora which are scarce. This paper aims to bridge this gap by developing an unsupervised method for acquiring word senses from readily available comparable corpora and bilingual dictionaries.

Methods

The core of the proposed method is Translingual Distributional Word Clustering. Unlike traditional methods that characterize words by their co-occurring words in the same language, this approach characterizes a word by its translation equivalents in another language.

Here's a breakdown of the key steps:

  1. Sense-vs.-Clue Correlation Matrix Calculation:
    • Alignment of Related Words: Pairs of related words (co-occurring within a window) are identified in both corpora.
  2. Translingual Alignment: These word pairs are then aligned across languages using a bilingual dictionary.
  3. Weighted Common Related Words: For each aligned pair, a weighted set of common related words is constructed. The weights are derived from mutual information, with a boost if the word is a translation of a word that itself has a translation in the other language.
  4. Correlation Calculation: A correlation matrix is computed between potential senses (defined by translation equivalents) and "clues" (words related to the target word). This calculation is iterative and designed to handle sparse data.
  5. Clustering of Translation Equivalents:
    • Subordinate Distribution Patterns: To address issues where a single translation equivalent might represent multiple senses, the method calculates correlation matrices not only for the full set of senses but also for sets of senses excluding one sense at a time. This helps to isolate the distributional patterns of individual senses.
  6. Sense Similarity: The similarity between senses is defined based on the similarity of their "subordinate" distribution patterns to the main distribution pattern of another sense. This similarity measure is asymmetric, reflecting dominance in the corpus.
  7. Hierarchical Merging: Similar senses are iteratively merged using a hierarchical agglomerative clustering approach until no further merging is deemed necessary based on a similarity threshold. This process aims to produce a hierarchy of corpus-relevant senses.

Datasets

  • Wall Street Journal (WSJ) corpus: 189 Mbytes.
  • Nihon Keizai Shimbun (Nikkei) corpus: 275 Mbytes. These are comparable corpora (similar domains, different languages).
  • EDR Bilingual Dictionary: A large dictionary containing 633,000 pairs of English-Japanese nouns.

The paper also mentions preparing a "standard sense inventory" for evaluation by defining an average of 3.4 senses for 60 target English words, which were then manually sense-tagged in 100 instances each from the WSJ corpus. This serves as a gold standard for evaluation.

Key Insights and Ideas

  • Leveraging Bilingualism for Sense Acquisition: The core idea is that by looking at how translation equivalents of a word are used in different linguistic contexts (as captured by their co-occurrence patterns in comparable corpora), we can infer their distinct meanings.
  • "Subordinate Distribution Patterns" for Disentangling Senses: The novel concept of calculating correlation matrices after excluding a sense is crucial. This allows the method to isolate the specific distributional signature of a sense, even when translation equivalents are polysemous, leading to cleaner clusters.
  • Corpus-Dependent Senses: The method naturally produces senses that are relevant to the specific domain of the comparable corpus, discarding irrelevant or rare senses. This is a significant advantage over generic, manually curated lexicons.
  • Unifying WSD and WSA: The method builds upon techniques originally developed for Word Sense Disambiguation (WSD), ensuring that the acquired senses are machine-distinguishable and allowing for automatic optimization of sense granularity.

Key Results

  • Recall of Senses: The method achieved high recall:
  • 96% for senses appearing in at least 25% of instances.
  • 87% for senses appearing in at least 5% of instances.
  • 78% for senses appearing in at least 1% of instances.
  • Accuracy of Sense Definitions: The average accuracy of the acquired sense definitions, compared to a standard inventory, was 77%.
  • Computational Efficiency: The method was found to be computationally acceptable, taking approximately 13 minutes per target word on a HP9000 C200 workstation (200 MHz CPU, 32 MB RAM) to produce a hierarchy of translation equivalent clusters.

Limitations and Future Work

  • Dictionary Dependency: The quality and coverage of the bilingual dictionary are critical. If a translation equivalent for a sense is missing from the dictionary, that sense cannot be acquired. The authors note the EDR dictionary is "fairly good" but an "exhaustive" one is ideal.
  • Low-Frequency/Non-Topical Senses: The method struggles with very low-frequency senses or senses that are not well-represented in the corpus (e.g., the sense of "bar" as a "legal profession" was acquired, but "piece of solid material" was not).
  • Hierarchy vs. Disjoint Clusters: The method produces a hierarchy of senses, not a set of strictly disjoint clusters. The authors acknowledge the difficulty in determining the optimal point to terminate merging in the hierarchy and suggest future work on methods to autonomously terminate merging, possibly by comparing complementary distribution patterns.
  • One Sense Per Translation Equivalent Assumption: The method assumes each translation equivalent represents only one sense of the target word, which is not always true, especially with transliterated words. Future work could involve detecting and splitting such equivalents.

Summary updated

BibTeX
@inproceedings{kaji2003word,
  author = {Hiroyuki Kaji},
  title = {{Word Sense Acquisition from Bilingual Comparable Corpora}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2003},
}
Figure