A Characterwise Windowed Approach to Hebrew Morphological Segmentation

Amir ZeldesEMNLP 2018

TL;DR

This paper introduces a novel character-wise approach to Hebrew morphological segmentation, treating it as a binary classification problem. By leveraging shallow, windowed features and lexicon lookups, the method achieves state-of-the-art accuracy on benchmark datasets, significantly outperforming previous methods, especially on out-of-domain data.

What Problem Does This Paper Address?

Hebrew, like Arabic, is a morphologically rich language where single "words" (delimited by spaces) often contain multiple meaningful units (morphemes). This complexity makes accurate segmentation a challenge. Traditional approaches often involve a two-step process: first, splitting by spaces, and then segmenting the resulting "super-tokens" into smaller units. However, Hebrew presents unique difficulties: it uses a consonantal script that omits most vowels, obscuring syllable structure crucial for segmentation. Additionally, some morphemes, like the definite article after certain prepositions, are not explicitly represented in the orthography, leading to ambiguity. This paper tackles the problem of pure orthographic segmentation – deciding where to split characters within a word form – without necessarily performing full morphological analysis or disambiguation.

Methods

The core of this paper's contribution is a character-wise binary classification approach. For each character within a "super-token" (a space-delimited word), the model predicts whether a boundary should follow it. This is achieved by extracting features from a window of characters around the target character, as well as from preceding and succeeding super-tokens. The key feature categories include:

  • Character Identity Features: The actual letters or punctuation marks within the window.
  • Numerical Position/Length Features: Information about the position of the character and the lengths of super-tokens.
  • Lexicon Lookup Features: This is a crucial component. The system queries a lexicon (derived from the Hebrew Treebank and expanded with proper names) to find possible Part-of-Speech (POS) tags for substrings within the window. This provides rich contextual information.

The authors experimented with various learning algorithms, ultimately finding a Random Forest variant (RFTokenizer) to be the most effective.

Datasets

The paper evaluates its approach on two datasets:

  • SPMRL Dataset: This is the benchmark dataset for Hebrew morphological segmentation, derived from the Hebrew Treebank. It was adapted for this study by removing inserted sub-tokens and reverting morphological analysis alterations.
  • Wiki5K Dataset: A new, out-of-domain dataset created from Hebrew Wikipedia data. This dataset consists of 5,000 super-tokens (7,100 sub-tokens) manually segmented to assess the model's robustness to domain shifts.

Key Insights and Ideas

  • Pure Segmentation is Valuable: The paper argues that achieving high accuracy in pure orthographic segmentation is valuable in itself, even without full morphological disambiguation. Such segmentations can be useful for downstream tasks like machine translation or can serve as input to more complex disambiguation systems.
  • Character-wise Classification is Robust: Treating segmentation as a character-wise binary classification problem, especially with windowed features, proves to be surprisingly robust, particularly for handling out-of-vocabulary (OOV) words. This is because it doesn't require a complete, coherent analysis of every word.
  • Lexicon Lookup is Central: The lexicon lookup features, which provide POS tag information for substrings, are shown to be the most critical component for achieving high performance.
  • Letter Identity is Less Crucial Than Lexicon: Surprisingly, the identity of individual letters is less critical than having access to lexicon lookup information. This suggests that the contextual information provided by the lexicon is more important than the specific characters themselves.
  • Vowel Features Offer Marginal Gains: Features indicating "vocalic" letters (used to represent vowels) provide a small but noticeable improvement, helping with generalizations, especially for affixes.

Key Results

The proposed RFTokenizer system achieved state-of-the-art performance:

  • SPMRL Dataset: Achieved 98.19% perfect segmentation and an F-score of 97.08%. This is a significant improvement over previous methods.
  • Wiki5K Dataset: Achieved 97.63% perfect segmentation and an F-score of 96.35%. This demonstrates strong robustness to out-of-domain data, outperforming other systems that degraded more substantially.
  • The system also showed high precision, which is crucial for reducing spurious segmentations.

Limitations and Future Work

  • No Reconstruction of Unexpressed Articles: The current approach does not reconstruct unexpressed articles, which is a common task in Hebrew NLP. The authors suggest this could be handled in a subsequent morphological analysis step.
  • DNN/Embedding Performance: While the DNN approach with embeddings was explored, it did not outperform the RFTokenizer. The authors speculate that more training data or optimized word representations might improve DNN performance in the future.
  • Lexicon Size and Coverage: While the lexicon was expanded, its coverage of foreign names is still partial. Further expansion could yield improvements.
  • Frequency Feature: The frequency feature, representing substring frequency ratios, had a small contribution and was retained more for historical reasons. Its impact is minimal.

Future work could involve exploring more advanced word representations optimized for segmentation and further expanding the lexicon.

Summary updated

BibTeX
@inproceedings{zeldes2018characterwise,
  author = {Amir Zeldes},
  title = {{A Characterwise Windowed Approach to Hebrew Morphological Segmentation}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2018},
}
Figure