Word-Sense Disambiguation for Machine Translation

David Vickrey, Luke Biewald, Marc Teyssier, Daphne KollerEMNLP 2005

TL;DR

This paper tackles the challenge of word-sense disambiguation (WSD) by reframing it as a "word translation" task, leveraging readily available bilingual corpora. They demonstrate that this approach significantly improves translation accuracy and can be effectively integrated into machine translation systems, offering a more practical and data-rich alternative to traditional WSD.

What Problem Does This Paper Address?

Word-sense disambiguation (WSD) aims to identify the correct meaning of a word in a given context. However, achieving high performance in WSD has been notoriously difficult due to several factors:

  • Fine-grained Senses: Ontologies like WordNet define very subtle distinctions between senses, making them hard to differentiate even for humans and requiring massive amounts of labeled data.
  • Data Scarcity: Obtaining sufficient hand-labeled data for WSD is a major bottleneck, especially for the "all-words" task where every ambiguous word in a corpus needs disambiguation.
  • Isolation from Application: Treating WSD as an isolated problem limits its practical utility. The paper argues that WSD is most valuable when applied to a larger goal, such as machine translation (MT).

The paper proposes that for MT, many of these fine-grained distinctions are irrelevant. Instead, the focus should be on identifying the correct translation of a word, which is often a more straightforward task with abundant training data.

Methods

The core innovation is reframing WSD as word translation. Instead of assigning a sense label, the system predicts possible translations of a source word into a target language.

  • Data Acquisition: The authors leverage bilingual corpora (like the European Parliament proceedings) to automatically create "partially-labeled" data. They use statistical alignment tools (GIZA++) to align sentences and then words, creating a corpus where source words are linked to their target language translations. This bypasses the need for manual sense labeling.
  • Algorithm: They adapt standard WSD algorithms, specifically using logistic regression, to the word translation task.
  • Features: The model uses features derived from the word's context within the sentence, including part-of-speech tags and binary "occurs" variables indicating the presence of words within a defined window around the target word. They also explore a generalized feature set that considers all possible context sizes.
  • Model: A logistic regression model is trained to predict the probability of a target translation given the source word and its context. This discriminative model is chosen for its ability to handle large datasets and produce confidence-rated predictions.
  • Integration into MT: The paper explores two ways to use their word translation model to improve MT:
  • Improving Translation Models: The output of the word translation model can help an MT system choose better words by providing ranked translation options.
  • Search Space Pruning: The model can be used to prune the set of candidate translations, significantly speeding up MT decoders.
  • Blank-Filling Task: To isolate and evaluate the impact of word translation on MT, they introduce a novel "blank-filling" task. Here, known translations of ambiguous words are replaced with blanks, and the system must fill them in. This decouples the evaluation from syntactic correctness and other MT complexities.

Datasets

  • European Parliament Proceedings Corpus: Used for training and evaluating the word translation models. This corpus contains approximately 27 million words in English and French.
  • Blank-Filling Benchmark: A new benchmark created from the Europarl corpus, consisting of 655 sentences with 3018 blanks representing ambiguous words.

Key Insights and Ideas

  • Word Translation as a Practical WSD: Reframing WSD as word translation unlocks access to vast amounts of naturally occurring, partially labeled data from bilingual corpora, overcoming the data scarcity problem of traditional WSD.
  • Context Matters for Translation: The paper demonstrates that incorporating contextual features significantly improves the accuracy of predicting word translations, outperforming simpler baselines.
  • Discriminative Models Shine with Data: Logistic regression, a discriminative model, performs well because the automatically acquired data is abundant enough to support its training, unlike traditional WSD where data is scarce.
  • Unified Approach to MT Improvement: The word translation model can be integrated into MT systems in multiple ways, either by directly influencing translation choices or by efficiently pruning the search space.
  • Blank-Filling Task as a Clean Evaluation: The blank-filling task provides a controlled environment to assess the contribution of word translation to MT quality, isolating it from other MT challenges.

Key Results

  • Word Translation Accuracy: The proposed logistic regression model achieved significantly higher accuracy on the word translation task compared to baselines. For example, for the word "rise," accuracy increased from 27.9% (part-of-speech baseline) to 57.0%. Micro-averaged accuracy reached 62.0% for the best model.
  • Blank-Filling Performance: The word translation model improved blank-filling accuracy by approximately 2.5% over the source-channel model and 1.3% over a model with generative and discriminative alignment models. This represents a relative error reduction of 14% and 7.8%, respectively.
  • Search Space Pruning Efficiency: The word translation model significantly outperformed discriminative and generative alignment models in pruning candidate translations. For 95% recall, it required an average candidate set size of 4.2 words, compared to 6.7 for the discriminative model.

Limitations and Future Work

  • Limitations:
    • The evaluation metric for word translation only rewards the exact translation used in the corpus, potentially leading to artificially low accuracies when multiple equally good translations exist.
  • The reliance on GIZA++ for alignment means the training data can be noisy due to potential alignment errors.
  • The Europarl corpus might not be "balanced" in terms of semantic content, the effect of which is unclear.
  • The current approach focuses on word-to-phrase translation and doesn't explicitly handle multi-word phrases as atomic units in the initial preprocessing.
  • Future Work:
    • Extend the model to handle phrases as atomic units.
  • Incorporate more sophisticated context features.
  • Explore different ways to integrate the word translation model into full MT systems, particularly with advanced decoders.
  • Investigate learning null fertility (when a word has no translation).

Summary updated

BibTeX
@inproceedings{vickrey2005word,
  author = {David Vickrey and Luke Biewald and Marc Teyssier and Daphne Koller},
  title = {{Word-Sense Disambiguation for Machine Translation}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2005},
}
Figure