What is Hidden among Translation Rules

Libin Shen, Bowen ZhouEMNLP 2013 · DOI

Introduction

In Statistical Machine Translation (SMT), systems like "Hiero" rely on massive tables containing tens of millions of translation rules (e.g., "how are you" $\rightarrow$ "cómo estás"). Traditionally, these rules are treated as discrete, unrelated atoms. While researchers have tried using "sparse features" (like specific word or Part-of-Speech tags) to bridge the gaps between rules, these features often fire randomly and fail to capture the structural relationships between different rules.

This research moves away from treating machine translation rules as isolated, independent events. Instead, the authors propose a "meta-grammar"—a compact hidden model that generates these rules—allowing the system to recognize underlying patterns and better generalize to translations not explicitly seen during training. By applying this to a Chinese-to-English system, they achieved a significant improvement in translation quality metrics.

The Meta-Grammar Approach

The authors identify a lack of generalization in current SMT models. If a system sees two rules that are structurally similar but lexically different, it treats them as completely separate. This paper asks: What if we could model the "DNA" that generates these rules?

To address this, the authors introduce a Hidden Model ($M$), which acts as a meta-grammar. Instead of just observing a rule $r$, they assume $r$ was generated by a sequence of "meta-rules."

Mapping of source and target POS tags in a meta-rule structure.

They propose three ways to integrate this hidden structure into a translation system:

  1. Type 1 (Generative): They calculate the log-likelihood of a rule being generated by the meta-grammar and use it as a single "dense" feature.
  2. Type 2 (Sparse Binary): They treat each meta-rule as a binary feature (1 if the meta-rule was used to generate the translation rule, 0 otherwise).
  3. Type 3 (Posterior): Instead of a binary 0 or 1, they use the posterior probability that a meta-rule was used, smoothed by a scaling factor ($\alpha$).

Experimental Setup

The researchers conducted a case study using Part-of-Speech (POS) tags. They converted standard translation rules into sequences of POS tags (e.g., "NR VV NR $\Rightarrow$ NNP VBZ IN NNP"). They then used the Expectation-Maximization (EM) algorithm to estimate the parameters of the hidden model that generates these POS-based meta-rules.

The approach was evaluated on a Chinese-to-English translation task using approximately 10 million parallel sentences from the DARPA BOLT task. The baseline was a hierarchical phrase-based system containing roughly 17 million rules. Performance was measured using BLEU (higher is better), TER (lower is better), and a combined TER-BLEU (T-B) metric (lower is better).

Key Results and Insights

The core "aha moment" of this work is that translation rules follow syntactic templates. By modeling these templates (meta-rules), the system can assign meaningful scores to rare rules that look like common templates.

Performance comparison on Test-2 (MT08-WB) across different system types.

As shown in the results for Test-2 (NIST MT08-WB), the best-performing models (Type 2 and Type 3) showed significant improvements over the baseline. Specifically:

  • Overall Improvement: The best models showed an improvement of approximately 1.0 point on the T-B metric over a strong baseline.
  • Tuning Beats Pure Generation: Interestingly, the "Type 1" generative model provided only marginal gains. The real power came from "Type 2" and "Type 3," where the system could discriminatively tune the weights of individual meta-rules. This suggests that while the generative model finds the patterns, the optimizer needs to decide which patterns actually matter for translation quality.
  • Smoothing Matters: For the posterior-based features (Type 3), using a very small scaling factor ($\alpha=0.01$) to prevent the model from being "too sharp" yielded the best results.

Conclusion and Future Work

This meta-grammar approach demonstrates that a compact model can effectively represent linguistic knowledge that is otherwise lost in massive rule tables. While the current case study relied on coarse POS tags, future work could explore unsupervised word splitting or more complex structures like Tree Adjoining Grammars (TAG) to capture more nuance. Additionally, the authors suggest exploring richer models that incorporate reordering and lexical information directly into the hidden state to further improve translation fidelity.

Summary updated

BibTeX
@inproceedings{shen2013what,
  author = {Libin Shen and Bowen Zhou},
  title = {{What is Hidden among Translation Rules}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2013},
  doi = {10.18653/v1/d13-1081},
}
Figure