Introduction
Statistical Machine Translation (SMT) systems, while robust, often struggle with the "long tail" of language—rare phrases, domain-specific jargon, and idiosyncratic expressions that lack sufficient statistical representation in massive training corpora. While Translation Memories (TM) offer high-precision alternatives, integrating them into "black-box" online MT services remains a challenge. This paper, originally presented at IJCNLP 2013, proposes a novel framework for "repairing" MT outputs by fusing them with retrieved bilingual examples at the word level.
System Architecture
The proposed system treats the output of an existing online MT system (e.g., Google Translate) as a "skeleton" or backbone. This skeleton provides a grammatically fluent starting point, which is then refined using specific knowledge retrieved from a private bilingual corpus.
The workflow consists of four primary stages: retrieval of similar examples, construction of a confusion network, feature-based confidence estimation, and final decoding.
Methodology
1. Retrieval and Alignment
For every input sentence, the system employs a word-based vector space model using Cosine Similarity to identify the top-15 most relevant sentence pairs from a bilingual example corpus. These retrieved target-side examples are then aligned against the MT "skeleton." This alignment is crucial as it allows the system to identify potential word-level substitutions or insertions.
2. Confusion Network Construction
A confusion network is constructed where the original MT output serves as the primary path. The retrieved examples are integrated as alternative arcs. This structure enables a word-level competition between the general statistical strengths of the MT system and the localized precision of the examples.
3. Feature-Rich Decoding
The final translation is produced by decoding the confusion network using a log-linear model. In addition to standard SMT features like Language Models, the authors introduce specialized features:
- Original Word Penalty: Regulates the degree of deviation from the initial MT skeleton.
- Repetitive Word Penalty: Mitigates common alignment artifacts where words are erroneously duplicated.
- Word Confidence & Posterior Probability: Uses GIZA++ alignments to ensure that words selected from examples are contextually relevant to the source input.
Experimental Evaluation
The approach was validated using the HIT dataset from the IWSLT 2012 Evaluation Campaign (Olympics task). The training corpus consisted of 52,603 Chinese-English sentence pairs, with a development set of 2,057 pairs and a test set of 998 pairs. Google Translate served as the baseline "Online MT System."
Results and Analysis
The "Repaired" system demonstrated a significant performance boost over the baseline. As shown in the table below, the fusion method outperformed both the original MT output and a simple replacement strategy.
Key Findings:
- Performance Gain: The repaired system achieved a BLEU score of 19.41, a +0.64 absolute improvement over Google Translate (18.77) and notably higher than the IWSLT 2012 best-reported score of 19.17.
- Fusion vs. Replacement: A "Replaced" baseline—which simply uses the most similar example as the translation—yielded a poor BLEU score of 10.22. This underscores that the value lies in the fusion of MT fluency with example-based precision.
- Similarity Sensitivity: Analysis revealed that the repair is most effective when example similarity scores fall between 0.7 and 0.8. When similarity is too low, the system correctly defaults to the original MT output to avoid introducing noise.
Conclusion
This research provides a practical and effective method for organizations to enhance generic MT services using their internal translation data. By allowing examples to compete at the word level within a skeleton-based confusion network, the system successfully bridges the gap between broad statistical coverage and domain-specific accuracy. Future work may explore more advanced retrieval techniques and improved alignment strategies to further minimize noise in low-similarity scenarios.
Summary updated