Extracting Morphophonology from Small Corpora

Marina ErmolaevaEMNLP 2018

TL;DR

This paper introduces a novel computational approach to extract phonological rules from small, annotated linguistic datasets, particularly useful for under-resourced languages. By combining information-theoretic methods with linguistic knowledge, the algorithm effectively identifies morphophonological alternations and their governing contexts, even for complex, non-local phenomena like vowel harmony.

What Problem Does This Paper Address?

Learning phonological rules typically relies on large, unannotated corpora. However, for many languages, especially those with limited digital resources, such large datasets are unavailable. Instead, linguists often collect small, hand-annotated datasets through fieldwork. This paper addresses the challenge of extracting meaningful phonological patterns, specifically morphophonology (the interface between morphology and phonology), from these small, annotated datasets. The core difficulty lies in inferring rules from limited surface forms, where a single morpheme can have multiple surface realizations (allomorphy) due to various phonological processes, including complex, non-local ones like vowel harmony.

Methods

The paper proposes a two-step algorithm that combines information-theoretic measures with linguistic constraints.

  1. Extraction: The algorithm first identifies potential alternations by treating groups of morphs (words or morphemes with the same gloss tag) as sets. It leverages the concept of "string differences" and "self-difference arrays" (inspired by group theory) to find variations within these sets. An "alternation" is defined as a set of segment-sized differences within a regular paradigm, with the constraint that these differences must be short.
  2. Reduction: Once potential alternations are identified, a reduction step collapses morph groups that are identical up to these known alternations. This process iterates, making more alternations accessible in subsequent passes, until the number of distinct groups stabilizes.

To determine the contexts for these alternations, the paper employs Mutual Information (MI) and Pointwise Mutual Information (PMI). These metrics quantify the statistical dependence between segments, helping to identify trigger segments (which cause the alternation) and transparent segments (which do not interfere). The algorithm explores four modes of collecting bigrams: local/nonlocal and left/right contexts.

Finally, the identified rules are evaluated for phonological viability, meaning that the trigger segments for an alternation should form disjoint natural classes. This ensures that the extracted rules are linguistically plausible.

Datasets

The primary dataset used is a hand-glossed collection of Mishar Tatar texts, elicited from native speakers during a linguistic expedition from 1999–2012. This dataset comprises 3090 word tokens and 1740 unique word types. The paper does not introduce new benchmark datasets but rather demonstrates its approach on this specific, small, annotated corpus.

Key Insights and Ideas

  • Bridging the Gap: The paper successfully demonstrates that information-theoretic methods can be effectively combined with linguistic knowledge to learn phonological rules from small datasets, a crucial step for under-resourced languages.
  • Beyond Local Processes: The approach is capable of identifying both local (adjacent segment influence) and nonlocal (distant segment influence, like vowel harmony) phonological processes, which is a significant challenge for many learning algorithms.
  • Context is Key: Mutual Information proves to be a powerful tool for assigning contexts (triggers and transparent segments) to alternations, moving beyond simple surface string matching.
  • Phonological Plausibility Matters: Incorporating the concept of "phonological viability" (natural classes) as a filter for generated rules is essential for ensuring linguistic interpretability and robustness.
  • Iterative Refinement: The iterative extraction and reduction process allows the algorithm to progressively uncover more complex patterns as simpler ones are accounted for.

Key Results

The algorithm was applied to Mishar Tatar, a language with agglutinative morphology and vowel harmony. The paper reports that the algorithm converged after three iterations, reducing 55 gloss tags and 160 surface realizations into 85 groups.

Specific results include:

  • Identification of several alternations, such as {d, n, t}, {d, t}, {Ø, k, g}, {l, n}, {k, g}, {Ø, r, e}, {a, ä}, and {e, r}.
  • The evaluation of these alternations using MI and "explained examples" (EE) metrics showed promising results, with some alternations achieving high scores for both left and right contexts. For instance, the {e, r} vowel harmony alternation achieved an EE of 0.7912 for right contexts.
  • Heat maps visualizing PMI values clearly illustrated the relationships between triggers and alternants, distinguishing between local and nonlocal processes.

Limitations and Future Work

The authors acknowledge several limitations and suggest future directions:

  • Interactions between Alternation Sets: The current approach treats alternations somewhat independently. Future work could explore how to decompose complex alternations (e.g., those involving both local and nonlocal processes) into simpler, constituent rules.
  • Data Noise and Completeness: The limited size of the dataset means that the learned rules might be noisy, and the algorithm may not have access to all possible contexts. The authors suggest that the learned rules could be used to identify potential mislabelling or inaccuracies in the dataset.
  • Natural Class Definition: While phonological viability is incorporated, the paper notes that the definition of natural classes could be further explored, especially for more complex segment sets.
  • Predicting Absent Morphs: The framework could be extended to predict and reconstruct morphs that are not present in the dataset.
  • Generalization: The approach is presented as a proof-of-concept, and further testing on a wider range of languages and phenomena is needed.

Summary updated

BibTeX
@inproceedings{ermolaeva2018extracting,
  author = {Marina Ermolaeva},
  title = {{Extracting Morphophonology from Small Corpora}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2018},
}
Figure