How Synchronous are Adjuncts in Translation Data?

Sophie Arnoult, Khalil Sima'anSSST @ EMNLP 2014 · DOI

TL;DR

This paper investigates how well the assumption of "synchronous adjunction" – where grammatical structures are translated in lockstep – holds up in real-world French-English translation data. By developing a novel method to identify and pair adjuncts across languages, even when they don't align perfectly, the authors find that only about 70% of manually identified adjuncts have clear translation equivalents, suggesting that synchronous translation models might be too rigid.

What Problem Does This Paper Address?

In computational linguistics, particularly in machine translation (MT), there's a concept called "adjunction" which refers to optional grammatical elements that modify a sentence. Theories like Synchronous Tree Adjoining Grammars (STAG) propose that these adjuncts should be handled "synchronously" during translation, meaning their counterparts in the target language should appear at the same structural level and at the same time as their source counterparts. This is known as the "Direct Correspondence Assumption." However, it's unclear how well this idealized assumption holds up in actual human translations, where linguistic structures can diverge and translations can be more or less literal. This paper aims to empirically test the degree of synchronous adjunction in parallel translation data, moving beyond previous studies that relied on limited resources or strict alignment assumptions.

Methods

The core of this paper is a new method for identifying and pairing adjuncts across a bitext (parallel sentences in two languages). Instead of relying solely on strict word alignments, the authors propose a "many-to-many" approach that allows for more flexible pairings.

Here's a breakdown of their method:

  • Adjunct Identification: They first identify adjuncts in dependency trees of English and French sentences using parser outputs and specific linguistic rules (e.g., mapping modifier labels to adjuncts, with exceptions for closed-class words).
  • Projection: For each adjunct in one language, they define its "projection" as the set of aligned phrases in the other language that it corresponds to via word alignments.
  • Many-to-Many Pairing: They then align adjuncts by projecting them through word alignments and finding the target adjuncts that maximally overlap with the source adjunct's projection. This process is done in both directions (English-to-French and French-to-English) to capture bidirectional correspondences. Crucially, this method allows for many-to-many pairings, meaning one adjunct in one language could correspond to multiple adjuncts in the other, or vice-versa.
  • Categorization of Pairings: They categorize these pairings into three main types:
  • Divergent: Adjuncts that don't have a clear translation equivalent.
  • Equivalent: Adjuncts that are translation equivalents and their alignment is consistent with word alignments (bijective).
  • Weakly Equivalent: Adjuncts that are translation equivalents but their alignment is not perfectly consistent with word alignments (e.g., many-to-many non-bijective, or where the alignment doesn't fully cover the projection).

Datasets

The study uses several French-English parallel corpora:

  • Canadian Hansards corpus: Both manually aligned (447 sentence pairs) and automatically aligned (1.13M sentence pairs).
  • Europarl corpus: Training data (1.97M sentence pairs).
  • Moses news-commentaries corpus: (156k sentence pairs).

For evaluating adjunct identification accuracy, they used a subset of 100 sentence pairs from the Europarl corpus.

Key Insights and Ideas

  • The "Direct Correspondence Assumption" is too strict: The paper strongly suggests that the ideal of synchronous, one-to-one correspondence for adjuncts in translation is not consistently met in real data.
  • Many-to-many pairings are crucial: Allowing for many-to-many alignments is essential for capturing translation equivalence, especially when dealing with parsing mismatches or more complex linguistic structures.
  • Adjunct labeling accuracy matters immensely: The difference in results between manually identified adjuncts (70% equivalence) and automatically identified adjuncts (around 40-50% equivalence) highlights the critical role of accurate linguistic annotation.
  • Word alignments are noisy and can be misleading: The study shows that relying solely on word alignments can lead to misclassifications of adjunct equivalence, and that more robust methods are needed.
  • Sentence length can impact alignment quality: Longer sentences tend to have more divergent or weakly equivalent adjunct pairings, likely due to increased alignment and parsing errors.

Key Results

  • Manually identified adjuncts: Approximately 70% of manually identified adjuncts have translation equivalents in the training data.
  • Automatically identified adjuncts: This figure drops to around 40-50%, with an additional 25% forming weakly equivalent pairings.
  • Impact of alignment: The study found that different word alignment tools (Berkeley aligner, GIZA++) and alignment strategies (manual vs. automatic) yield varying proportions of adjunct pairing types.
  • Sentence length effect: Longer sentences show a higher proportion of many-to-many and weakly equivalent pairings, suggesting increased alignment and parsing errors.

Limitations and Future Work

  • Adjunct Identification Limitations: The authors acknowledge that their adjunct identification method, while improved, is not perfect. Errors in parsing or the heuristic rules used can lead to misclassified adjuncts, impacting the overall results. They note that about a quarter of errors are related to parse attachment.
  • Future Work on Divergence: The paper raises the question of why a significant portion of adjuncts have divergent translations. Future work could investigate the causes of this divergence, such as lexical conflation or non-literal translation strategies.
  • Improving Alignments: The authors suggest that further gains in measuring adjunct equivalence could be obtained with better word alignment techniques.
  • Cross-linguistic Divergence: While this study focuses on French-English, future research could explore how synchronous adjunction holds up in language pairs with greater structural differences.

Summary updated

BibTeX
@inproceedings{arnoult2014how,
  author = {Sophie Arnoult and Khalil Sima'an},
  title = {{How Synchronous are Adjuncts in Translation Data?}},
  booktitle = {SSST @ EMNLP},
  year = {2014},
  doi = {10.3115/v1/w14-4019},
}
Figure