Integration of a Multilingual Preordering Component into a Commercial SMT Platform

Anita Ramm, Riccardo Superbo, Dimitar Sht. Shterionov, Tony O'Dowd, Alexander M. FraserPrague Bull. Math. Linguistics 2017 · OpenReview

Abstract

This paper describes the integration of a high-speed, rule-based preordering component into the KantanMT commercial Statistical Machine Translation (SMT) platform. By aligning English source syntax with target languages like German, Japanese, and Chinese before translation, the system achieves significant quality improvements in SMT. However, the study also highlights a critical divergence: while preordering benefits phrase-based SMT, it consistently degrades the performance of Neural Machine Translation (NMT) models, likely due to the introduction of parser-induced noise that NMT's global encoding is sensitive to.

Introduction

Machine translation (MT) performance often degrades when source and target languages exhibit divergent word orders, such as English (SVO) and Japanese (SOV). In Statistical Machine Translation, these long-range reorderings are difficult to capture during the standard phrase-alignment process. Preordering—rearranging the source sentence to mirror the target language's syntax—offers a solution. While well-researched in academia, implementing preordering in a commercial environment requires balancing translation quality with processing speed, scalability, and traceability for engineering maintenance.

Methodology

The researchers developed a deterministic preordering pipeline designed for high-throughput production environments. The process involves three primary stages:

  1. Parsing: The English source text is processed using the Stanford Shift-Reduce (SR) parser. This parser was selected for its superior speed compared to traditional PCFG parsers, which is essential for commercial scalability.
  2. Tsurgeon-based Reordering: Tree-to-tree transformations are applied using Tsurgeon. Hand-crafted linguistic rules identify specific syntactic structures (e.g., relative clauses) and move them to positions consistent with the target language's grammar.
  3. Read Out: The reordered parse tree is flattened back into a string, which serves as the modified input for the translation engine.
The preordering pipeline: from source sentence parsing to Tsurgeon-based tree manipulation and final readout.

To optimize for industry use, the system utilizes GNU Parallel to distribute tasks across multiple CPU cores. Additionally, the parser is maintained as a persistent web service to avoid the overhead of reloading models for every request.

Experimental Setup and Results

The system was evaluated using legal domain corpora for English to German (~1M sentences), Japanese (~213k sentences), and Chinese (~387k sentences). Performance was measured using BLEU, TER, and F-Measure.

SMT Performance

Preordering yielded substantial improvements in SMT quality across all tested language pairs. The most notable gains were observed in English→Japanese (+1.89 BLEU) and English→German (+1.39 BLEU). These results validate the effectiveness of aligning syntactic structures for phrase-based models.

Comparative performance metrics (TER, F-measure, BLEU) across different parsers and language pairs.

The NMT Contrast

In a significant finding, the researchers observed that preordering negatively impacts Neural Machine Translation. For English→German, NMT BLEU scores dropped from 38.26 to 36.74 when preordering was applied. Unlike SMT, which operates on local phrases, NMT models (specifically RNN encoder-decoders) encode the entire sentence into a continuous vector space, making them naturally more robust to word order variations. The manual reordering step appears to introduce "noise"—errors from the parser—that disrupts the neural model's internal representations.

Computational Efficiency

The parallelized implementation demonstrated high efficiency. Processing 5,000 segments, which took 263 seconds in a serial configuration, was reduced to just 46 seconds using 8 CPU cores.

Discussion and Conclusion

The integration of preordering into KantanMT demonstrates that rule-based syntactic alignment remains a powerful tool for enhancing commercial SMT, particularly for distant language pairs. However, the "double-edged sword" nature of preordering in the context of NMT suggests that as the industry shifts toward neural architectures, the focus must move from manual preordering to improving the robustness of neural encoders or integrating syntax more directly into the neural architecture. Future work will explore whether these findings persist with Transformer-based models and if "noising" techniques can mitigate the negative impact of parser errors on NMT.

Summary updated

BibTeX
@inproceedings{ramm2017integration,
  author = {Anita Ramm and Riccardo Superbo and Dimitar Sht. Shterionov and Tony O'Dowd and Alexander M. Fraser},
  title = {{Integration of a Multilingual Preordering Component into a Commercial SMT Platform}},
  booktitle = {Prague Bull. Math. Linguistics},
  year = {2017},
  url = {https://openreview.net/forum?id=uFKQL30UoE},
}
Figure