Towards Unsupervised and Language-independent Compound Splitting using Inflectional Morphological Transformations

Patrick Ziering, Lonneke van der PlasNAACL 2016

TL;DR

This paper introduces a novel, unsupervised method for splitting complex compound words in Germanic languages. By leveraging inflectional morphology learned from lemmatized corpora, their approach significantly outperforms existing language-dependent methods in accurately identifying compound boundaries and normalizing constituent parts, all without requiring parallel data.

What Problem Does This Paper Address?

Many languages, especially Germanic ones like German and Dutch, heavily rely on compounding to create new words. Think of "Armutsbekämpfungsprogramm" (poverty elimination program). While these compounds are incredibly productive, they pose a significant challenge for Natural Language Processing (NLP) tasks like machine translation and information retrieval. To process these words effectively, we need to:

  1. Split them correctly: Identify the individual words that form the compound (e.g., "Armut" | "Bekämpfung" | "Programm").
  2. Normalize the parts: Reduce these parts to their base form (lemma), which is often tricky because compounds can involve "linking elements" (like the 's' in "Armuts") or other morphological changes that don't appear in the base word.

Previous methods often relied on language-specific knowledge, large lexicons, or parallel corpora, limiting their applicability and robustness. This paper aims to overcome these limitations by developing a truly language-independent and unsupervised approach.

Methods

The core innovation lies in approximating compounding morphology using inflectional morphology. The authors hypothesize that the way words change when inflected (e.g., pluralization, genitive case) shares many similarities with how they change when forming compounds.

Here's a breakdown of their approach:

  • Language-Independent Splitting: The system doesn't hardcode rules for specific languages.
  • Unsupervised Learning: It learns from raw, lemmatized monolingual corpora without needing pre-annotated compound data.
  • Inflectional Morphology as a Proxy: They extract "Morphological Operation Patterns" (MOPs) from lemmatized corpora. These MOPs describe the transformations (insertions, deletions, replacements) from a lemma to its inflected form.
  • Ngram Index for Candidate Retrieval: To find potential lemmas for a constituent part, they use an Ngram index. This allows for "tolerant string retrieval," meaning it can find similar strings even if they aren't an exact match, accounting for minor differences.
  • Scoring and Ranking: Candidate lemmas are scored based on two main features:
  • Lemma Prominence (LP): How frequent the lemma is in the corpus and how often it appears as a candidate for the given constituent.
  • MOP Suitability (MS): How well the inflectional MOP (learned from the corpus) transforms the candidate lemma into the constituent part, considering edit distance.
  • Recursive Splitting: The splitter works recursively, first identifying binary splits and then refining them until atomic constituents are reached.
  • Rightmost Head Rule (RHHR): They assume that the head of a compound (the main semantic and POS component) is the rightmost constituent, a common linguistic principle.

Datasets

The experiments were conducted on the following corpora:

  • German: Wikipedia corpus (665M words, 9.0M types)
  • Dutch: Wikipedia corpus (114M words, 2.0M types)
  • Afrikaans: Taalkommissie corpus (57M words, 748K types)

For evaluation, they used existing gold standards:

  • German: GermaNet binary split compound set (51,230 samples)
  • Dutch & Afrikaans: Split point gold standards developed by Verhoeven et al. (2014) (21,941 for Dutch, 17,369 for Afrikaans).

Key Insights and Ideas

  • Inflection is a powerful proxy for compounding: The paper demonstrates that the morphological changes seen in word inflection are surprisingly effective at approximating the changes involved in compounding, even for complex cases. This is a significant departure from methods relying on explicit compounding rules.
  • Language-independent and unsupervised is achievable: By focusing on general morphological principles and leveraging readily available monolingual data, the authors show that high performance can be achieved without language-specific lexicons or parallel corpora.
  • Separate evaluation of splitting and normalization is crucial: The paper highlights that these are distinct tasks and that performance on one doesn't guarantee performance on the other. Normalization, in particular, is shown to be a harder discipline.
  • Ngram index for tolerant retrieval is effective: Using Ngrams allows the system to find candidate lemmas even when there are minor string differences, which is essential for handling the variations introduced by compounding.

Key Results

  • German Performance: The proposed system (LP.MSinfl) significantly outperforms previous state-of-the-art methods like Fritzinger & Fraser (2010) and Weller & Heid (2012) in split point accuracy (SPAcc) and normalization accuracy (NormAcc), especially for top-ranked splits (@2 and @3). For example, it achieves 99.4% SPAcc@3 and 96.5% NormAcc@3.
  • Comparison to Baselines: The system also performs competitively against a hand-crafted set of MOPs (LP.MSLanger) and an upper bound using gold-standard MOPs (LP.MSGS), suggesting that inflectional MOPs are a strong approximation.
  • Dutch and Afrikaans: The system also shows strong performance on Dutch and Afrikaans, outperforming supervised methods for Dutch split point prediction.
  • Normalization is Harder: Across all systems, normalization accuracy is consistently lower than split point accuracy, underscoring the difficulty of this task.

Limitations and Future Work

  • Approximation Limitations: The authors acknowledge that inflectional morphology is not a perfect approximation for compounding in all languages, but it works well for Germanic languages, which are rich in compounding and often require unsupervised methods.
  • Future Work: The authors plan to adapt their methods for languages with special compound markers, such as Greek. They also aim to explore more sophisticated ways to learn compounding morphology.

This research offers a significant step towards more robust and widely applicable compound splitting tools, paving the way for better NLP performance across a wider range of languages and applications.

Summary updated

BibTeX
@inproceedings{ziering2016towards,
  author = {Patrick Ziering and Lonneke van der Plas},
  title = {{Towards Unsupervised and Language-independent Compound Splitting using Inflectional Morphological Transformations}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2016},
}
Figure