Modeling Syntactic and Semantic Structures in Hierarchical Phrase-based Translation

Junhui Li, Philip Resnik, Hal Daumé IiiNAACL 2013 · OpenReview

Abstract: Incorporating semantic structure into a linguistics-free translation model is challenging, since semantic structures are closely tied to syntax. In this paper, we propose a two-level approach to exploiting predicate-argument structure reordering in a hierarchical phrase-based translation model. First, we introduce linguistically motivated constraints into a hierarchical model, guiding translation phrase choices in favor of those that respect syntactic boundaries. Second, based on such translation phrases, we propose a predicate-argument structure reordering model that predicts reordering not only between an argument and its predicate, but also between two arguments. Experiments on Chinese-to-English translation demonstrate that both advances significantly improve translation accuracy.

Introduction

Hierarchical phrase-based (HPB) translation models, such as the "Hiero" system, utilize synchronous context-free grammars to manage nested structures and long-distance reordering. While formally syntactic, these models are often linguistically naive because their rules are derived from statistical word alignments rather than formal grammar trees. This often results in translations that, while containing the correct vocabulary, suffer from grammatical errors or "semantic role confusion," such as swapping subjects and objects. This paper addresses these issues by injecting linguistic structure into the HPB framework without sacrificing the flexibility of statistical models.

Methodology

The authors propose a two-level approach to integrate linguistic knowledge into the HPB framework: relaxed syntactic constraints and a novel Predicate-Argument Structure (PAS) reordering model.

Relaxed Syntactic Constraints

Previous attempts to enforce strict syntactic constraints often failed because they were too rigid, filtering out useful non-constituent translation phrases. To solve this, the authors allow phrases that span "sibling subtrees"—multiple children under the same parent node. Furthermore, they "flatten" deep parse trees so that a head verb and its modifiers appear at the same level, making the constraints flexible enough to guide the model without being overly restrictive.

Flattened parse tree for the English sentence

Predicate-Argument Structure (PAS) Reordering

The core semantic contribution is a PAS reordering model based on PropBank-style semantic roles. This model calculates the probability of how a predicate and its arguments should be ordered in the target language. To address data sparsity, the model decomposes reordering into two components: predicate-argument reordering (how an argument moves relative to its verb) and argument-argument reordering (how two arguments move relative to each other).

Source-side Predicate-Argument Structure (PAS-S)

Additionally, the authors introduce reorderable glue rules. Unlike standard HPB models that stitch translated chunks monotonically (left-to-right), these rules allow the decoder to swap the order of chunks if the grammar suggests a better alignment.

Experimental Setup

The system was evaluated on a Chinese-to-English translation task using 1.5 million sentence pairs from various LDC datasets. The researchers used a 5-gram language model trained on the Xinhua portion of the Gigaword corpus. Evaluation was conducted on NIST MT 2002, 2004, and 2005 test sets, with the Berkeley Parser providing syntactic analysis and a specialized labeler for semantic roles.

Key Results and Insights

  • Performance Gains: The integrated system (Syntactic Constraints + Reorderable Glue Rules + PAS Model) achieved an average improvement of 1.03 BLEU points over the standard HPB baseline. Syntactic constraints alone improved scores from 36.10 to 36.71, while adding the PAS model further increased the score to 37.13.
  • Efficiency: Applying syntactic constraints significantly reduced the search space. Rule extraction dropped from 193 million to 60 million rules, making the decoder faster and more memory-efficient.
  • Semantic Integrity: The PAS model was particularly effective for long, complex sentences (20+ words), where baseline models often lose track of "who-did-what" relationships.

Conclusion

By balancing linguistic rigidity with statistical flexibility, the authors demonstrate that both syntax and semantics can be effectively integrated into hierarchical translation. Future work may address the current model's reliance on source-side syntax and the sparsity of rare predicates by utilizing verb classes or target-side syntactic information during decoding.

Summary updated

Abstract

Incorporating semantic structure into a linguistics-free translation model is challenging, since semantic structures are closely tied to syntax. In this paper, we propose a two-level approach to exploiting predicate-argument structure reordering in a hierarchical phrase-based translation model. First, we introduce linguistically motivated constraints into a hierarchical model, guiding translation phrase choices in favor of those that respect syntactic boundaries. Second, based on such translation phrases, we propose a predicate-argument structure reordering model that predicts reordering not only between an argument and its predicate, but also between two arguments. Experiments on Chinese-to-English translation demonstrate that both advances significantly improve translation accuracy.
BibTeX
@inproceedings{li2013modeling,
  author = {Junhui Li and Philip Resnik and Hal Daumé III},
  title = {{Modeling Syntactic and Semantic Structures in Hierarchical Phrase-based Translation}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2013},
  url = {https://openreview.net/forum?id=BkWP6GW_ZS},
}
Figure