Abstract
Recently system combination has been shown to be an effective way to improve translation quality over single machine translation systems. In this paper, we present a simple and effective method to systematically derive an ensemble of SMT systems from one baseline linear SMT model for use in system combination. Each system in the resulting ensemble is based on a feature set derived from the features of the baseline model (typically a subset of it). We will discuss the principles to determine the feature sets for derived systems, and present in detail the system combination model used in our work. Evaluation is performed on the data sets for NIST 2004 and NIST 2005 Chinese-to-English machine translation tasks. Experimental results show that our method can bring significant improvements to baseline systems with state-of-the-art performance.
Introduction
In Statistical Machine Translation (SMT), system combination is a proven technique for enhancing translation quality by leveraging the consensus among multiple translation engines. Traditionally, this requires developing several distinct systems (e.g., phrase-based vs. syntax-based), which is resource-intensive. This paper introduces the Feature Subspace Method, a lightweight approach to generate a diverse ensemble of "expert" sub-systems from a single baseline model by selectively manipulating its feature set.
The Feature Subspace Method
Most SMT models utilize a linear framework where various features—such as word translation probabilities, phrase penalties, and language models—are weighted and summed to score translations. The proposed method derives sub-systems through two primary techniques:
- Sub-system Generation: Creating model variants by excluding a single non-language-model feature (e.g., removing the word penalty) or "weakening" the Language Model (LM) by reducing its n-gram order (e.g., from a 5-gram to a 3-gram).
- Independent Tuning: Each derived sub-system undergoes independent Minimum Error Rate Training (MERT) to optimize weights for its specific feature subspace.
- Sentence-Level Combination: A combination model pools $n$-best translation lists from all systems and uses n-gram consensus to select the final output, rewarding agreement while penalizing length-based discrepancies.
Experimental Results
The method was evaluated on Chinese-to-English translation tasks using the NIST 2004 (MT04) and NIST 2005 (MT05) test sets. The training corpus consisted of 5.1 million sentence pairs.
As shown in Figure 1, individual sub-systems typically perform slightly worse than the full baseline. However, the diversity they introduce is crucial. The combined system (FS-Comb) achieved gains of 1.1 to 1.3 BLEU points over the baseline. Furthermore, the "Oracle BLEU" score improved by approximately 1.5 points, indicating that the ensemble successfully captured high-quality translations that the baseline missed.
Qualitative Analysis and Diversity
The effectiveness of the Feature Subspace Method stems from the fact that training data can be noisy, leading to poorly estimated features for certain sentences. A sub-system lacking a misleading feature may produce a superior translation for specific contexts.
Analysis revealed that 60-70% of the candidates suggested by sub-systems were unique relative to the baseline's top results. Figure 5 illustrates a case where the sub-system correctly identified "sea water," a detail omitted by the default system, demonstrating how feature deletion can mitigate the impact of noisy estimations.
Conclusion
The Feature Subspace Method provides a cost-effective way to boost SMT performance without the overhead of building multiple distinct architectures. By forcing the decoder to operate in different feature subspaces, the method generates a wide variety of high-quality candidate translations that can be effectively combined to surpass the baseline.
Summary updated