Inferring Inflection Classes with Description Length

Sacha Beniamine, Olivier Bonami, Benoît SagotJ. Lang. Model 2017 · DOI · OpenReview

How do we objectively group words into "inflection classes"—like verb conjugations or noun declensions—without relying on subjective intuition? This question is at the heart of a recent study published in the Journal of Language Modelling, which introduces an algorithmic approach to discovering these linguistic structures.

The Challenge of Macroclasses

In many languages, words change their form based on their grammatical role, a process known as inflection. Linguists typically distinguish between two levels of organization:

  1. Microclasses: Groups of words that behave identically in every form.
  2. Macroclasses: Broader groups of words that share similar patterns, even if they aren't identical (e.g., the "-er" verbs in French).

While microclasses are mathematically straightforward to define, macroclasses are often "fuzzy." Traditional grammar books might list a handful of conjugations, but these are frequently based on historical tradition or pedagogical convenience rather than a rigorous, data-driven standard. The gap addressed by this research is the lack of a systematic way to decide which words belong together and exactly how many groups a language "actually" has.

Traditional Latin noun declensions as an example of inflectional macroclasses.

Methodology: Inflection as Compression

The authors treat the problem of classification as a compression problem, utilizing the Minimum Description Length (MDL) principle. MDL posits that the "best" model is the one that results in the shortest total description of the data. In this context, it balances two competing costs:

  • The cost of the rules: A simpler grammar with fewer macroclasses is easier to describe.
  • The cost of the exceptions: If there are too few macroclasses, many words will deviate from their group's "standard" pattern, increasing the cost of describing those deviations.

The algorithm uses a bottom-up "greedy search" clustering method. It starts with every word in its own class and repeatedly merges the two classes that result in the greatest reduction of the total description length.

Key Insights: Local vs. Global Strategies

A significant finding of the study was the superiority of a "local" strategy over a "global" one.

  • The Global Strategy looks at the entire set of forms for a word simultaneously. However, this approach proved too sensitive to minor, unique irregularities, causing the system to fragment into many tiny, unhelpful classes.
  • The Local Strategy compares pairs of forms (e.g., 1st person singular vs. 2nd person singular). This pairwise approach produced much more linguistically sensible results, as it focuses on shared sub-patterns rather than total identity.

Results in French and Portuguese

The method was evaluated on large-scale, phonemically transcribed datasets of French and European Portuguese verbs. Using phonemic transcriptions (how words sound) allowed the algorithm to capture real morphological alternations that standard spelling often masks.

  • French: The local strategy inferred 6 macroclasses. This aligns well with the traditional "three groups" of French verbs but provides a more nuanced breakdown of the complex "third group" of irregular verbs.
  • Portuguese: The algorithm identified 5 macroclasses, corresponding to the three traditional conjugations (-ar, -er, and -ir) plus two distinct groups of highly irregular verbs.

Conclusion: Objective Traditionalism

One of the most compelling takeaways is that the algorithm, starting with zero prior knowledge, naturally arrived at classifications very similar to traditional grammar. This suggests that traditional linguistic categories are not merely arbitrary conventions; they represent the most efficient way to "compress" and learn a language's structure. Furthermore, the results show that the "regular/irregular" binary is too simplistic. "Irregular" verbs often share significant sub-patterns with "regular" ones, and the MDL approach places them in macroclasses based on where they do fit, rather than simply labeling them as outliers.

Summary updated

Abstract

We discuss the notion of an inflection class system, a traditional ingredient of the description of inflection systems of nontrivial complexity. We distinguish systems of microclasses, which partition a set of lexemes in classes with identical behavior, and systems of macroclasses, which group lexemes that are similar enough in a few larger classes. On the basis of the intuition that macroclasses should contribute to a concise description of the system, we propose one algorithmic method for inferring macroclasses from raw inflectional paradigms, based on minimisation of the description length of the system under a given strategy for identifying morphological alternations in paradigms. We then exhibit classifications produced by our implementation on French and European Portuguese conjugation data, and argue that they constitute an appropriate systematisation of traditional classifications. To arrive at such a concincing systematisation, it is crucial though that we use a local approach to class similarity (based on pairwise comparisons of paradigm cells) rather than a global approach (based on simultaneous comparison of all cells). We conclude that it is indeed possible to infer inflectional macroclasses objectively.
BibTeX
@article{beniamine2017inferring,
  author = {Sacha Beniamine and Olivier Bonami and Benoît Sagot},
  title = {{Inferring Inflection Classes with Description Length}},
  journal = {J. Lang. Model.},
  year = {2017},
  doi = {10.15398/jlm.v5i3.184},
}
Figure