Abstract
Graham Neubig. Proceedings of the 1st Workshop on Asian Translation (WAT2014).
2014.
Introduction
Translating between Japanese and languages like English or Chinese presents a significant structural challenge. Japanese follows a Subject-Object-Verb (SOV) order, while English and Chinese are Subject-Verb-Object (SVO). Standard phrase-based machine translation (PBMT) often fails to capture these long-distance reorderings because it lacks a deep understanding of sentence structure.
At the 2014 Workshop on Asian Translation (WAT), the NAIST team addressed this gap using syntax-based translation. Specifically, they moved beyond traditional tree-to-string models—which are often brittle to single parsing errors—to a more robust "forest-based" approach.
Methodology
The NAIST system, built on the open-source Travatar decoder, utilizes several key techniques:
- Forest-to-String (F2S) Translation: Instead of relying on a single "best" parse tree, the system generates a "packed forest" representing many possible syntactic analyses. This allows the decoder to select the most likely translation even if the top-ranked parse was incorrect.
- RNNLM Rescoring: To improve fluency, the team used a Recurrent Neural Network Language Model (RNNLM). They generated the top 10,000 translation candidates and used the RNNLM’s scores to re-rank them.
- Supervised Alignment: For English-Japanese pairs, they utilized Nile, a syntax-based aligner that learns from manually created alignments, providing cleaner data than standard unsupervised methods.
- Targeted Preprocessing: Techniques included compound splitting for Japanese and vocabulary normalization to reduce domain mismatch.
Experimental Setup
The primary data source was the ASPEC corpus provided by the WAT task. For the zh-ja/ja-zh pairs, 672k sentences were used. For en-ja/ja-en, the team filtered the translation data to the cleanest 2 million sentences. External dictionaries (EDICT, Eijiro) and Wikipedia links were incorporated for the ja-en track to minimize unknown word issues.
Key Results
The NAIST system achieved the highest results in both automatic (BLEU/RIBES) and manual evaluations across all four language pairs. The transition from PBMT to F2S provided massive gains, particularly in the en-ja pair where BLEU scores increased by over 8 points.
As shown in the evaluation metrics, the addition of RNNLM consistently provided a 0.7 to 1.0 BLEU point boost. In human evaluations (H), the system frequently outperformed the best competing systems, maintaining correct meaning in complex sentences.
Discussion and Key Insights
- Syntax is Essential for Distant Languages: The significant gain from F2S over PBMT confirms that modeling underlying grammar is more effective than local word sequences for languages with radically different word orders.
- The "Forest" Buffer: Using a forest of parses makes the system resilient to the inherent errors of automated parsers.
- Metric Tension (BLEU vs. RIBES): Tuning for RIBES (which rewards word order) often resulted in shorter, "clipped" sentences. While these scored well on RIBES, they sometimes performed worse in manual evaluations because they dropped information. Tuning for BLEU generally produced more faithful translations.
Conclusion
The NAIST system's success at WAT 2014 demonstrates the power of combining forest-to-string SMT with neural rescoring. Future work will focus on improving Japanese parsing accuracy, which remains a major bottleneck, and exploring more sophisticated parameter tuning methods.
Summary updated