Multiword Expressions in the Context of Statistical Machine Translation

Mahmoud Ghoneim, Mona T. DiabIJCNLP 2013 · OpenReview

Introduction

Multiword Expressions (MWEs)—such as idioms ("kick the bucket"), compound nouns ("traffic light"), and phrasal verbs ("write up")—represent idiosyncratic concepts that cross word boundaries. In Statistical Machine Translation (SMT), systems typically treat language as sequences of individual words or arbitrary phrases. This often leads to suboptimal translations because the system fails to recognize MWEs as single semantic units. This paper investigates how explicitly modeling different types of MWEs can improve English-to-Arabic SMT, achieving gains of up to 0.82 BLEU points.

Methodology and Integration Strategies

The researchers categorized MWEs based on their syntactic flexibility: fixed expressions (no variation), semi-fixed expressions (non-decomposable idioms and proper names), and syntactically-flexible expressions (phrasal verbs). To integrate these into a phrase-based SMT framework (Moses), four strategies were tested:

  1. Static Integration (S): MWEs are concatenated with underscores (e.g., traffic_light) during preprocessing, treating them as single vocabulary tokens.
  2. Dynamic Integration (D): A "soft constraint" feature is added to the translation model to bias the system toward translations that preserve MWE integrity without altering the tokens.
  3. Zone Integration (Z): XML tags are used to force the decoder to respect MWE boundaries.
  4. Hybrid Integration: A combination of methods tailored to specific MWE types.
Table 4: Statistics of MWE schemes used in the experiments, including lemma and token counts.

Experimental Setup

The study utilized a parallel corpus from the LDC GALE newswire (English-Arabic) consisting of approximately 474,000 sentence pairs. MWEs were sourced from WordNet 3.0 and the Stanford NER tool. Evaluation was conducted on NIST MT04 (development), MT05, and MT08 (blind test) sets.

Results and Discussion

The experimental results demonstrate that the optimal integration strategy depends heavily on the MWE type. Static integration is most effective for rigid adverbial and adjectival phrases. However, for nouns and named entities, static integration can lead to data sparsity and "Out of Vocabulary" (OOV) errors. In these cases, dynamic integration performs better by keeping words separate while encouraging their cohesive translation.

Table 3: Performance comparison of various MWE integration strategies across different evaluation metrics and test sets.

The hybrid approach (A_VAA-D_NE+NN), which uses MWEs to improve initial word alignment and applies dynamic features for Named Entities and Nouns, yielded the most consistent improvements. Specifically, it achieved an absolute BLEU score increase of 0.82 on the MT05 test set and 0.45 on MT08.

Conclusion

Explicitly modeling MWEs significantly enhances translation quality, even though they represent a small fraction of total tokens. The study highlights that a "one-size-fits-all" approach is insufficient; integration must be tailored to the linguistic properties of the MWE. Future work will address MWEs with gaps (e.g., "break it up") and explore MWE modeling in the target language.

Summary updated

BibTeX
@inproceedings{ghoneim2013multiword,
  author = {Mahmoud Ghoneim and Mona T. Diab},
  title = {{Multiword Expressions in the Context of Statistical Machine Translation}},
  booktitle = {IJCNLP},
  year = {2013},
  url = {https://openreview.net/forum?id=mYT5GJjJOy},
}
Figure