Phrase-Based SMT with Shallow Tree-Phrases

Philippe Langlais, Fabrizio GottiNAACL 2006 · OpenReview

Abstract

Philippe Langlais, Fabrizio Gotti. Proceedings of the Workshop on Statistical Machine Translation, 2006. This paper explores a middle ground between "flat" phrase-based translation and complex syntax-based systems. By introducing "Tree-Phrases," the authors attempt to capture non-contiguous linguistic patterns that standard models often miss.

1. Introduction

In the mid-2000s, Phrase-Based Statistical Machine Translation (PB-SMT) was the dominant paradigm. By treating sequences of words as atomic translation units, PB-SMT systems successfully captured local reorderings and idiomatic expressions. However, two significant limitations persisted: the inability to handle non-contiguous patterns (such as the French negation ne ... pas) and a lack of mechanisms to model global structural differences between languages.

While some researchers pursued full "synchronous parsing"—treating translation as a simultaneous grammar exercise for two languages—these systems were computationally expensive and often relied on the assumption of structural isomorphism between languages. Langlais and Gotti (2006) proposed a simpler alternative: Tree-Phrases (TPs). This approach injects shallow syntactic information into the translation process without the overhead of full tree-to-tree mapping.

2. The Tree-Phrase Framework

The core innovation of this work is the Tree-Phrase, a translation unit that pairs a source-language dependency structure with a flexible target-language representation.

2.1 Source Treelets (TL)

A Tree-Phrase begins with a Source Treelet (TL), which is a small, lexicalized subtree extracted from a dependency parse of the source sentence. To maintain computational efficiency and reduce data sparsity, the authors limited these treelets to a "shallow" depth of 1, consisting of a governor and its immediate dependents.

A full dependency tree for the French sentence "on a demandé des crédits fédéraux," providing the structural context for treelet extraction.

As shown in the figure above, a full dependency parse provides a rich set of relationships. The extraction process breaks this complex structure down into simpler, manageable units.

The decomposition of the full dependency tree into two distinct depth-1 treelets: one centered on the verb "a_demandé" and another on the noun "crédits."

2.2 Elastic Phrases (EP)

The target-language component of a TP is an Elastic Phrase (EP). Unlike standard phrases, the words in an EP do not need to be adjacent in the target sentence. Instead, their relative positions (offsets) are recorded. This "elasticity" allows the model to capture non-contiguous translations naturally, addressing one of the primary weaknesses of standard PB-SMT.

3. System Architecture and Decoding

The authors implemented their approach by modifying the Pharaoh decoder, a standard tool in the SMT community at the time. The translation pipeline involves several key steps:

  1. Parsing: Source text is processed using the SYNTEX dependency parser.
  2. Extraction and Alignment: Depth-1 treelets are extracted and aligned with target words using GIZA++ word-alignment matrices.
  3. Compatibility-Based Decoding: During decoding, the system does not merely concatenate phrases. It checks if new treelets are "compatible" with the partial dependency structure already built for the sentence.
  4. Log-Linear Scoring: The system uses a model with nine features, including separate translation models for standard phrases and tree-phrases. This allows the system to balance the influence of syntactic units against traditional flat phrases.

4. Experimental Evaluation

The system was tested on the French-English Canadian Hansards, a large corpus of parliamentary proceedings. The training set comprised approximately 1.7 million sentence pairs.

The experimental results demonstrated that the TP-ENGINE (integrating Tree-Phrases) outperformed the baseline PP-ENGINE (standard phrase-based model). The TP-ENGINE achieved a BLEU score of 30.47, a gain of +0.52 over the baseline's 29.95. While seemingly modest, this improvement was statistically significant at the 95% confidence level. Furthermore, improvements in Word Error Rate (WER) and Sentence Error Rate (SER) were significant at the 99% level.

Analysis of the development set revealed that Tree-Phrases could cover roughly 70% of source sentences, indicating that even shallow syntactic structures have broad applicability across diverse linguistic contexts.

5. Conclusion and Future Directions

Langlais and Gotti's work demonstrates that shallow syntax can provide a meaningful boost to translation quality without the complexity of full syntactic parsing. By focusing on depth-1 treelets and elastic target phrases, they successfully captured non-contiguous patterns that elude standard PB-SMT.

Future research directions identified by the authors include exploring deeper treelets to capture more complex dependencies, implementing "fuzzy" matching to increase the utility of extracted treelets, and projecting more formal syntactic constraints onto the target language to improve grammatical fluency.

Summary updated

BibTeX
@inproceedings{langlais2006phrase,
  author = {Philippe Langlais and Fabrizio Gotti},
  title = {{Phrase-Based SMT with Shallow Tree-Phrases}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2006},
  url = {https://openreview.net/forum?id=mCnYOSG6wZu},
}
Figure