Abstract
System combination is a powerful technique for improving machine translation (MT) quality by merging outputs from multiple engines. Traditional methods rely on multi-step pipelines that are prone to error propagation. This paper proposes a joint optimization framework that integrates word alignment, ordering, and lexical selection into a single decoding step using a log-linear model. Evaluated on the NIST 2008 Chinese-to-English task, the proposed method significantly outperforms state-of-the-art confusion-network-based baselines, achieving a BLEU score of 37.20.
What Problem Does This Paper Address?
Machine translation systems often have complementary strengths and weaknesses. System combination aims to produce a "consensus" translation that is superior to any individual input. The standard approach utilizes Confusion Networks (CN), which typically follow a rigid pipeline: selecting a "backbone" system to define word order, aligning other systems to it, and finally selecting the best words.
This "greedy" pipeline suffers from the Backbone Bottleneck. If the chosen backbone contains errors or if the initial alignment is flawed, these mistakes propagate through the system. Furthermore, the alignment process is decoupled from the final lexical selection, preventing the system from making globally optimal decisions.
Methods: Joint Optimization Framework
The authors introduce a framework that treats system combination as a unified search task. Instead of a pipeline, they employ a single log-linear model to evaluate candidate translations generated via a joint decoding process.
Joint Decoding and Features
The system uses a beam search algorithm to construct the output sentence from left to right. At each step, the decoder considers various word choices and their alignments across all input hypotheses. The candidates are scored based on a set of feature functions:
- Word Posterior & Bi-gram Voting: Measures the degree of agreement among input systems for specific words or sequences.
- Alignment Model: Utilizes HMM-based scores to estimate the probability that words from different systems are translations of the same source concept.
- Distortion Model: Penalizes excessive reordering to maintain the natural flow of the hypotheses.
- Entropy: Quantifies the "purity" of word groupings, favoring alignments where systems are in high agreement.
By integrating these features into a single objective function, the decoder can balance the trade-offs between word choice, alignment quality, and fluency.
Key Insights and Ideas
Backbone-Free Search
By moving away from a fixed backbone, the system avoids the structural biases of any single engine. In traditional networks, if two systems agree on a word (e.g., "Jeep") but the backbone uses a different term (e.g., "SUV"), the agreement might be lost. Joint optimization allows words to find their optimal partners dynamically during the search.
Alignment as a Feature
Rather than committing to a "hard" alignment early on, the system treats HMM alignment probabilities as features. This allows the decoder to remain "uncertain" about specific alignments until more information (such as language model scores or voting patterns) becomes available.
Adjacency Constraints
To manage the exponential search space created by joint optimization, the authors introduced adjacency constraints. These constraints require the decoder to process input hypotheses in a relatively orderly fashion, ensuring computational feasibility without sacrificing significant translation quality.
Key Results
The approach was evaluated on the NIST 2008 Chinese-to-English translation task using 1,357 sentences and outputs from 5 high-performing MT systems. The joint decoding method demonstrated a clear advantage over both individual systems and established combination baselines.
| System | BLEU Score |
|---|---|
| Best Individual System | 32.03 |
| Standard IHMM Baseline | 35.85 |
| Incremental HMM Baseline | 36.38 |
| Joint Decoding (Proposed) | 37.20 |
The improvement of 0.82 BLEU points over the best baseline was found to be statistically significant at the 99% confidence level.
Limitations and Future Work
While effective, joint optimization is computationally intensive, requiring aggressive pruning and constraints. Future research will focus on improving "future score" heuristics to guide the beam search more efficiently. Additionally, the authors plan to extend the framework to handle N-best lists from each input system and incorporate more sophisticated features, such as syntax-based models, to further enhance translation quality.
Summary updated