Cohesive Phrase-Based Decoding for Statistical Machine Translation

Colin CherryACL 2008 · OpenReview

Abstract

Phrase-based decoding produces state-of-the-art translations with no regard for syntax. We add syntax to this process with a cohesion constraint based on a dependency tree for the source sentence. The constraint allows the decoder to employ arbitrary, non-syntactic phrases, but ensures that those phrases are translated in an order that respects the source tree’s structure. In this way, we target the phrasal decoder’s weakness in order modeling, without affecting its strengths. To further increase flexibility, we incorporate cohesion as a decoder feature, creating a soft constraint. The resulting cohesive, phrase-based decoder is shown to produce translations that are preferred over non-cohesive output in both automatic and human evaluations.

Introduction

In Statistical Machine Translation (SMT), phrase-based decoding has long been the dominant paradigm. It operates by translating contiguous segments of text—often called "phrases" in a non-linguistic sense—from left to right. While effective at capturing local context and idiomatic expressions, this approach struggles with long-distance word reordering, also known as distortion. Standard phrase-based systems typically penalize any deviation from a strict linear order with a flat cost, failing to account for the underlying linguistic structure of the sentence.

Figure 1: Mapping source dependency trees to translation phrases.

Conversely, syntactic SMT models use parse trees to guide movement but often lack the flexibility and memorization power of phrase-based models. This research proposes a middle ground: injecting structural awareness into the phrase-based framework via syntactic cohesion. The goal is to ensure that subtrees in the source sentence's parse tree remain contiguous in the target translation, acting as a structural guardrail for the decoder.

Methods: The Cohesion Constraint

The core innovation is a "syntactic cohesion" constraint integrated into the Moses phrase-based decoder. This constraint checks whether a translation hypothesis respects the boundaries of syntactic constituents in the source dependency tree.

Defining and Tracking Cohesion

Because SMT phrases often span multiple linguistic units or only parts of them, the system uses a relaxed metric called "span intersection." This identifies when a translation choice unambiguously interrupts a syntactic constituent. As the decoder builds a hypothesis, a linear-time algorithm monitors whether adding a new phrase "interrupts" an active subtree from the source parse.

Figure 2: Comparison of (a) a cohesive translation and (b) a non-cohesive translation where the 'voting' constituent is displaced.

Soft vs. Hard Constraints

Initial experiments with a hard constraint—rejecting any non-cohesive hypothesis—proved too restrictive. Parser errors or genuine linguistic divergences between English and French can make non-cohesive translations necessary. Consequently, the researchers implemented a soft constraint. The number of cohesion "interruptions" is treated as a weighted feature in the decoder's log-linear model, with weights optimized via Minimum Error Rate Training (MERT).

Experimental Setup

The model was trained on 688,000 English-French sentence pairs from the Europarl dataset. English source sentences were parsed using the Minipar dependency parser. Evaluation was conducted on a test set of 2,000 sentences, focusing on English-to-French translation.

Results and Discussion

The study found that the baseline phrase-based decoder naturally produces cohesive translations approximately 80% of the time. Interestingly, these naturally cohesive translations achieved significantly higher BLEU scores than non-cohesive ones, suggesting that syntactic violations are often indicators of translation errors.

Quantitative Performance

On the full test set, the soft cohesion constraint yielded a modest improvement of 0.2 to 0.5 BLEU points. However, the impact was much more pronounced on the "difficult" 20% of sentences where the baseline failed to maintain cohesion. In these cases, the cohesion-constrained model improved performance by 0.5 to 1.1 absolute BLEU points.

Human Evaluation

A blind human evaluation of the difficult subset confirmed these findings. Annotators preferred the cohesive output in 47% of cases, while the baseline was preferred only 8% of the time (the remaining 45% were considered equal). This indicates that the cohesion constraint effectively prevents the "scrambled" syntax that often plagues standard phrase-based systems.

Conclusion

By incorporating syntactic cohesion as a soft feature, phrase-based SMT can benefit from structural guidance without losing its inherent flexibility. While limited by parser accuracy and specific dependency representations, this approach provides a robust mechanism for improving translation quality, particularly for sentences requiring complex reordering. Future work may explore more flexible dependency representations to better handle phenomena like conjunctions.

Summary updated

Abstract

Phrase-based decoding produces state-of-theart translations with no regard for syntax. We add syntax to this process with a cohesion constraint based on a dependency tree for the source sentence. The constraint allows the decoder to employ arbitrary, non-syntactic phrases, but ensures that those phrases are translated in an order that respects the source tree’s structure. In this way, we target the phrasal decoder’s weakness in order modeling, without affecting its strengths. To further increase flexibility, we incorporate cohesion as a decoder feature, creating a soft constraint. The resulting cohesive, phrase-based decoder is shown to produce translations that are preferred over non-cohesive output in both automatic and human evaluations.
BibTeX
@inproceedings{cherry2008cohesive,
  author = {Colin Cherry},
  title = {{Cohesive Phrase-Based Decoding for Statistical Machine Translation}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2008},
  url = {https://openreview.net/forum?id=H1-c-pxuWB},
}
Figure