Handling Unknown Words in Statistical Latent-Variable Parsing Models for Arabic, English and French

Mohammed Attia, Jennifer Foster, Deirdre Hogan, Joseph Le Roux, Lamia Tounsi, Josef van GenabithSPMRL @ NAACL 2010 · OpenReview

Authors: Mohammed Attia, Jennifer Foster, Deirdre Hogan, Joseph Le Roux, Lamia Tounsi, Josef van Genabith
Venue: Proceedings of the NAACL HLT 2010 First Workshop on Statistical Parsing of Morphologically-Rich Languages

Abstract: This paper demonstrates that incorporating morphological "signatures"—clues like suffixes, prefixes, and word templates—significantly improves the accuracy of statistical parsers when they encounter unknown words. While the benefits are marginal for English, they are substantial for morphologically rich languages like Arabic and French. Furthermore, these clues can be learned automatically using Information Gain, offering a language-agnostic path to improved parsing performance.

Introduction

In natural language processing, a parser's primary task is to determine the grammatical structure of sentences. A significant challenge for statistical parsers, such as the Berkeley Latent-Variable PCFG parser (PCFG-LA), is the presence of "unknown words"—vocabulary not encountered during the training phase. While English-centric research often employs a simple "one-size-fits-all" approach by mapping rare words to a single UNKNOWN token, this strategy is insufficient for morphologically rich languages (MRLs) like Arabic and French.

The Challenge of Morphologically Rich Languages

In languages with low inflectional complexity, such as English, unknown words are relatively rare (approximately 3% in the Wall Street Journal corpus). However, in MRLs, words are heavily inflected; a single root can produce dozens of distinct forms. This results in a much higher "unknown" rate in test data. Treating a complex Arabic verb and a simple noun as the same generic token discards critical syntactic information, leading to poor parsing performance.

Figure 1: Data statistics across languages and frequency cutoffs, highlighting the higher percentage of unknown words in Arabic compared to English.

As illustrated in Figure 1, the percentage of unknown words is notably higher for Arabic than for English across various frequency cutoffs. The authors also created "reduced" versions of the datasets to confirm that the difficulty arises from linguistic complexity rather than merely the size of the training data.

Methodology

The researchers compared three distinct strategies for handling unknown words:

  1. Baseline: All words occurring below a specific frequency threshold are mapped to a single UNKNOWN symbol.
  2. Hand-Crafted Signatures: Linguistic heuristics categorize unknown words based on their appearance. For English and French, this focused on common suffixes. For Arabic, the authors developed 72 heuristics to identify "templates"—internal structures that distinguish nouns from verbs in non-concatenative morphology.
  3. Automatic Signatures: To create a language-agnostic solution, the authors used Information Gain (IG) to automatically rank and select the most predictive affixes from the training data to serve as signatures.

Key Results

The inclusion of morphological signatures significantly improved parsing accuracy, particularly for MRLs. Accuracy was measured using the F-score, which balances precision and recall in identifying grammatical structures.

Figure 2: Parsing performance with morphological signatures. The figures in parentheses indicate the F-score improvement over the baseline.

As shown in Figure 2, the impact of morphological clues is uneven across languages. While English saw a marginal improvement (+0.17% F-score), Arabic experienced a substantial boost of up to +4.18% depending on the frequency threshold. Notably, the automatic IG-based signatures performed nearly as well as the hand-crafted rules, achieving an 81.33% F-score on Arabic, trailing the manual rules by only 0.26%.

Conclusion

This study underscores that morphology is a vital component of statistical parsing for complex languages. While simple unknown-word handling may suffice for English, MRLs require more sophisticated approaches that leverage internal word structure. The success of automatically learned signatures suggests that parsers can be effectively adapted to new languages with minimal manual linguistic engineering.

Summary updated

BibTeX
@inproceedings{attia2010handling,
  author = {Mohammed Attia and Jennifer Foster and Deirdre Hogan and Joseph Le Roux and Lamia Tounsi and Josef van Genabith},
  title = {{Handling Unknown Words in Statistical Latent-Variable Parsing Models for Arabic, English and French}},
  booktitle = {SPMRL @ NAACL},
  year = {2010},
  url = {https://openreview.net/forum?id=Wud9cIaABiG},
}
Figure