Language Independent Connectivity Strength Features for Phrase Pivot Statistical Machine Translation

Ahmed El Kholy, Nizar Habash, Gregor Leusch, Evgeny Matusov, Hassan Sawaf2013

TL;DR

This paper tackles the challenge of low-quality translations in statistical machine translation (SMT) when using a "pivot" language to bridge data-scarce language pairs. They introduce novel "connectivity strength" features that measure the quality of phrase alignments through the pivot language, leading to a significant improvement in translation quality for Persian-Arabic.

What Problem Does This Paper Address?

Statistical Machine Translation (SMT) systems thrive on large amounts of parallel text (sentences and their translations). However, for many language pairs, this data is scarce or non-existent. A common workaround is "pivoting," where a third, data-rich language (like English) is used as an intermediary. For example, to translate from Persian to Arabic, one might translate Persian to English, and then English to Arabic. While effective, this "phrase pivoting" method can introduce many low-quality phrase translations because the connections between source and target phrases are indirect and mediated by the pivot. This leads to a massive, noisy phrase table, making decoding difficult and degrading overall translation quality. The authors aim to improve the quality of these pivot-based translations by better assessing the reliability of the phrase pairs generated through the pivot.

Methods

The core contribution of this paper is the introduction of two new language-independent features: Source Connectivity Strength (SCS) and Target Connectivity Strength (TCS). These features are designed to quantify the quality of the projected alignments between source and target phrases in the pivot phrase table.

Here's how they work:

  • Projection of Alignments: The method leverages word alignments from the source-pivot and pivot-target phrase tables. These alignments are "projected" to infer alignments between the original source and target phrases. If a source-target phrase pair is connected through multiple pivot phrases, the union of alignments is considered.
  • Connectivity Strength Calculation:
    • SCS: Measures the proportion of source words in a phrase pair that have at least one alignment link to a target word. It's calculated as the number of alignment links divided by the number of source words.
  • TCS: Measures the proportion of target words in a phrase pair that have at least one alignment link to a source word. It's calculated as the number of alignment links divided by the number of target words.
  • Integration into SMT: These SCS and TCS scores are then incorporated as additional features into the log-linear model of the SMT system. This allows the decoder to consider the "connectivity strength" of a phrase pair when making translation decisions, effectively down-weighting noisy or unreliable pivot-generated phrase pairs.

The paper also discusses baseline methods, including sentence pivoting and phrase pivoting with various filtering thresholds, which are used for comparison.

Datasets

  • Parallel Corpora:
    • English-Arabic: Approximately 2.8 million sentences (≈60 million words) from LDC and GALE constrained data.
  • Persian-English: An in-house corpus of about 170,000 sentences (≈4 million words).
  • Language Model Data:
    • Arabic: 200 million words from the Arabic Gigaword Corpus.
  • English: English Gigaword Corpus.
  • Evaluation Set: An in-house Persian-Arabic evaluation set of 536 sentences with three references.

Key Insights and Ideas

  • The "Noise" in Pivot Translation: The paper clearly articulates that while pivoting is a necessary technique for low-resource languages, it inherently introduces a significant amount of low-quality phrase pairs into the translation model. This is a critical bottleneck.
  • Alignment as a Quality Indicator: The core insight is that the density and coverage of word alignments between source and target phrases (even when projected through a pivot) can serve as a strong indicator of translation quality. A phrase pair with many cross-lingual links is likely more reliable than one with few.
  • Language Independence is Key: The proposed SCS and TCS features are language-independent. This is crucial because it means the approach can be readily applied to any language pair where pivoting is used, without requiring language-specific linguistic knowledge or preprocessing.
  • Complementary to Existing Features: Connectivity strength features provide information that is complementary to existing SMT features like lexical weights and phrase translation probabilities. They offer a more direct measure of alignment quality.

Key Results

  • Phrase Pivoting Outperforms Sentence Pivoting: The paper demonstrates that phrase pivoting, even with basic filtering, generally outperforms sentence pivoting.
  • Filtering Improves Performance: Increasing the filtering threshold for phrase pairs in phrase pivoting leads to better translation quality (measured by BLEU and METEOR scores), though with diminishing gains.
  • Connectivity Strength Boosts Quality: The most significant result is the improvement achieved by adding the connectivity strength features to the best-performing phrase pivoting model. This resulted in an increase of approximately 0.6 BLEU points and 0.3 METEOR points for Persian-Arabic translation. These improvements were statistically significant.

Limitations and Future Work

  • Limitations:
    • The paper acknowledges the "very large size of the induced phrase table" as a primary challenge in phrase pivoting, which their filtering and feature engineering aim to mitigate.
  • The quality of the projected alignments is dependent on the quality of the underlying source-pivot and pivot-target alignments.
  • Future Work:
    • Explore other language-independent features, such as the number of pivot phrases used in connecting a source-target pair and the similarity between these pivot phrases.
  • Investigate language-specific features that could be extracted from seed parallel data, focusing on syntactic and morphological compatibility between source and target phrase pairs.

Summary updated

Abstract

An important challenge to statistical machine translation (SMT) is the lack of parallel data for many language pairs. One common solution is to pivot through a third language for which there exist parallel corpora with the source and target languages. Although pivoting is a robust technique, it introduces some low quality translations. In this paper, we present two language-independent features to improve the quality of phrase-pivot based SMT. The features, source connectivity strength and target connectivity strength reflect the quality of projected alignments between the source and target phrases in the pivot phrase table. We show positive results (0.6 BLEU points) on Persian-Arabic SMT as a case study. 1
BibTeX
@misc{kholy2013language,
  author = {Ahmed El Kholy and Nizar Habash and Gregor Leusch and Evgeny Matusov and Hassan Sawaf},
  title = {{Language Independent Connectivity Strength Features for Phrase Pivot Statistical Machine Translation}},
  year = {2013},
}
Figure