A Unified and Discriminative Soft Syntactic Constraint Model for Hierarchical Phrase-based Translation

Lemao Liu, Tiejun Zhao, Chao Wang, Hailong CaoMTSummit 2011 · OpenReview

Introduction

Hierarchical phrase-based translation (the "Hiero" model) has become a cornerstone of statistical machine translation (SMT) due to its ability to capture long-distance reordering through synchronous context-free grammars. However, Hiero is primarily "formally" syntactic; while it follows a grammar-like structure, it often lacks an understanding of actual linguistic syntax, such as the boundaries of noun phrases or verb phrases. Researchers from the Harbin Institute of Technology address this by proposing a unified model that integrates linguistic "common sense" into Hiero without the typical performance overhead.

The Challenge of Soft Syntactic Constraints

Previous attempts to incorporate linguistic syntax into Hiero via "soft syntactic constraints" have faced two primary obstacles. First, many models rely on heuristic rigidity, using simple binary features that fail to account for the surrounding linguistic context. Second, standard MT optimization techniques, such as Minimum Error Rate Training (MERT), struggle to handle a large number of sparse features. Furthermore, treating all syntactic categories equally—regardless of their frequency—often introduces a model bias that degrades translation quality.

Proposed Methodology: Discriminative SSC

The authors introduce a Discriminative Soft Syntactic Constraint (SSC) model designed to overcome these hurdles. The model moves beyond simple binary flags by building sophisticated classifiers for various General Constituent Labels (GCL).

Decomposed Labels

A key innovation is the use of "decomposed" labels. Instead of only considering full constituents (e.g., Noun Phrase, NP), the model utilizes partial labels (e.g., L\NP for a phrase missing a component on the left) to provide finer-grained syntactic information to the translation engine.

Decomposed syntactic labels (GCL) for a Chinese phrase.

Discriminative Classifiers and Bayes-style Unification

The researchers trained Maximum Entropy (MaxEnt) and Logistic Regression (LogReg) models to predict the probability that a specific rule application is syntactically valid. These classifiers utilize features such as neighboring words, POS tags, and phrase lengths.

To avoid the "MERT wall," the authors developed a "secret sauce": a Bayes-style unification method. Instead of adding a separate feature for every syntactic type to the optimizer, they combined them into a single unified feature. Each classifier's output is weighted by the prior probability (frequency) of that syntactic constituent in the training data. This ensures that rare categories do not disproportionately skew the translation results.

Experimental Setup

The model was evaluated on the following datasets:

  • Training: The FBIS corpus (~190,000 Chinese-English sentence pairs).
  • Development: NIST02 evaluation data.
  • Testing: NIST05 Chinese-to-English translation task.
  • Parsing: The Stanford Parser generated gold-standard trees for the source (Chinese) side to extract training examples for the classifiers.

Key Insights and Results

The proposed unified model significantly outperformed both the baseline Hiero system and previous heuristic-based syntactic constraint methods.

System BLEU-4 Score
Baseline (Hiero) 27.45
XP+ (Heuristic) 27.85
MaxEnt SSC 28.35
Unified Combination 28.48

The improvement of +1.03 BLEU over the baseline is statistically significant. The authors also found that classifier accuracy is a strong predictor of translation quality; their "Combination" classifier (88% accuracy) outperformed standalone MaxEnt (83%) and LogReg (75%) models.

Priority vs. Uniformity

A critical finding was that accounting for the natural frequency of linguistic structures is vital. Treating all syntactic types as equal (Uniform) was less effective than the "With Priority" method.

BLEU-4 score comparison between uniform and priority-based unification.

As shown in the results, the "With Priority" method outperformed the "Uniform" combination by 0.65 BLEU points, validating the Bayes-style unification approach.

Conclusion and Future Work

The Discriminative SSC model demonstrates that linguistic syntax can be effectively integrated into hierarchical MT systems through careful classifier design and statistical unification. While successful, the authors noted limitations, such as the reliance on bilingual parsing (which succeeded for only 70% of training sentences) and the focus on source-side syntax. Future work may involve incorporating target-side (English) syntactic constraints and scaling the model to multi-million sentence datasets.

Summary updated

BibTeX
@inproceedings{liu2011unified,
  author = {Lemao Liu and Tiejun Zhao and Chao Wang and Hailong Cao},
  title = {{A Unified and Discriminative Soft Syntactic Constraint Model for Hierarchical Phrase-based Translation}},
  booktitle = {MTSummit},
  year = {2011},
  url = {https://openreview.net/forum?id=0TTyTO9OhFi},
}
Figure