Translation Acquisition Using Synonym Sets

Daniel Andrade, Masaaki Tsuchida, Takashi Onishi, Kai IshikawaNAACL 2013

TL;DR

This paper tackles the challenge of acquiring translations for words that are not well-represented in comparable corpora. They propose a novel method that leverages synonyms of a query term to create a more robust context vector, significantly improving translation accuracy compared to existing smoothing techniques.

What Problem Does This Paper Address?

Automatic translation acquisition is crucial for tasks like updating bilingual dictionaries and cross-lingual information retrieval. A common approach involves finding translations by comparing the context vectors of words in comparable corpora. However, a major hurdle arises when a query term is infrequent, leading to a "sparse" context vector with many missing features. This sparsity makes it difficult to reliably determine its meaning and find accurate translations. Existing methods try to "smooth" these sparse vectors, but often by using nearest neighbors, which can introduce noise if those neighbors have different meanings. This paper addresses the problem of improving translation acquisition accuracy for infrequent terms by using synonyms to create a more reliable representation of the query term's meaning.

Methods

The core of the proposed method lies in how it combines context vectors. Instead of relying on a single query term's context vector, the authors propose using a set of synonyms for that term. They assume that each occurrence of a word and its synonyms can be seen as samples from an underlying probabilistic model. To represent the combined meaning, they calculate the mean vector of this distribution. Specifically, they model this using the von Mises-Fisher distribution, a natural choice for directional data like normalized vectors. The combined context vector is then a weighted average of the individual context vectors of the query term and its synonyms, where the weights are determined by the frequency of each term. This weighted average is then L2-normalized. This approach is a departure from previous methods that might simply average vectors or use nearest neighbors for smoothing.

Datasets

The experiments were conducted using:

  • Japanese and English WordNet: Used for creating a gold-standard of aligned synsets to evaluate translation accuracy.
  • Comparable Corpora:
    • Japanese corpus: Complaints concerning automobiles from the Japanese Ministry of Land, Infrastructure, Transport and Tourism (MLIT).
  • English corpus: Complaints concerning automobiles from the USA National Highway Traffic Safety Administration (NHTSA).
  • Bilingual Dictionary: A dictionary with approximately 1.6 million entries was used to map English context vectors to the Japanese vector space.

Key Insights and Ideas

  • Synonyms as a powerful tool for disambiguation and robustness: Leveraging synonyms significantly improves translation acquisition, especially for infrequent terms, by providing a richer and more stable representation of meaning.
  • Von Mises-Fisher distribution as a suitable model: This distribution is well-suited for modeling directional data and proved effective for combining context vectors in this probabilistic framework.
  • Frequency-based weighting is key: Giving more weight to the context vectors of more frequent synonyms leads to better results, reflecting the assumption that frequent terms provide more reliable information.
  • Nearest neighbor smoothing can introduce noise: The paper experimentally confirms that smoothing with synonyms outperforms smoothing with nearest neighbors, highlighting the potential for meaning drift when using semantically distant but contextually similar neighbors.

Key Results

The proposed method, "Syn Mises-Combination," demonstrated significant improvements in translation accuracy.

  • It achieved a Top1 accuracy of 0.31, which is substantially higher than "No Smoothing" (0.14), "Top-k Smoothing" (0.16), and "Syn Smoothing" (0.18).
  • The Mean Inverse Rank (MIR) for "Syn Mises-Combination" was 0.40, outperforming all other baseline methods.
  • A pair-wise comparison showed a statistically significant improvement (p < 0.01) over "Syn Smoothing."
  • Even a simpler method of just summing synonym vectors ("Syn Sum") performed better than "Syn Smoothing," suggesting the inherent benefit of using synonyms.
  • In a qualitative example with the Japanese terms クルーズ and 巡航 (both meaning "cruise"), the proposed method retrieved the correct translation "cruise" at rank 2, compared to rank 10 and 15 for the baseline methods.

Limitations and Future Work

  • Ambiguity within Synsets: The paper acknowledges that even within a single synset, terms can have slightly different nuances or be conflated. While their method improves results in such cases, it doesn't fully resolve all ambiguity.
  • Reliance on existing resources: The method relies on the availability of WordNet for synset alignment and a bilingual dictionary for vector space mapping.
  • Future Work: The authors suggest that exploring other methods for combining context vectors or investigating alternative probabilistic models could be fruitful. They also hint at the potential of using canonical correlation analysis to match vectors to a common latent space, as described in other research.

Summary updated

BibTeX
@inproceedings{andrade2013translation,
  author = {Daniel Andrade and Masaaki Tsuchida and Takashi Onishi and Kai Ishikawa},
  title = {{Translation Acquisition Using Synonym Sets}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2013},
}
Figure