Implicational Universals in Stochastic Constraint-Based Phonology

Giorgio MagriEMNLP 2018 · DOI

Introduction

A central goal of phonological theory is to understand the "logic" of human language—specifically, the implicational universals that determine which linguistic patterns are possible. In traditional constraint-based frameworks like Optimality Theory (OT) and Harmonic Grammar (HG), these universals are captured through categorical mappings. However, the move toward stochastic models to account for linguistic variation raises a critical question: does the introduction of probabilistic "noise" preserve the underlying typological structure of the theory?

This paper explores the stability of these implicational universals, known as T-orders, when transitioning from categorical to stochastic frameworks. The findings suggest a surprising degree of robustness in standard models, while highlighting fundamental differences in others.

The Problem of Stochastic Variation

In categorical phonology, an underlying representation maps to a single surface form. In reality, speakers often exhibit variation, such as the optional deletion of final consonants. To model this, linguists use stochastic versions of OT and HG, which assign probabilities to various outputs.

While categorical typologies are finite and relatively straightforward to analyze, stochastic typologies are infinite, as they depend on continuous parameters like constraint weights or ranking values. This complexity makes it difficult to determine if a stochastic model makes the same universal predictions as its categorical counterpart. For instance, if a categorical model predicts that "Language X implies Language Y," does a noisy version of that model still guarantee that the probability of X is always bounded by the probability of Y?

T-Orders and the "Sufficient Richness" Condition

The author addresses this by focusing on T-orders—partial orders that represent entailments between linguistic mappings. The paper's primary contribution is a formal proof demonstrating that the T-orders of stochastic models converge with those of categorical models under a condition called "sufficient richness."

A stochastic framework is considered "sufficiently rich" if it can approximate any categorical grammar with arbitrarily high probability. If the noise distribution (such as Gaussian noise in Stochastic OT) allows the "signal" of the constraint rankings to dominate when necessary, the implicational logic remains intact.

Stability in SOT and SHG

The paper applies this proof to two dominant stochastic frameworks:

  1. Stochastic OT (SOT): Where constraint rankings are perturbed by noise.
  2. Stochastic HG (SHG): Where constraint weights are perturbed by noise.

The results show that SOT and SHG are "conservative" extensions of OT and HG, respectively. Despite the introduction of gradient probabilities, the set of implicational universals remains identical. In the "basic syllable system" (Prince and Smolensky, 2004)—a benchmark involving constraints like ONSET and NOCODA—all four models (OT, HG, SOT, SHG) agree on a core set of 16 entailments. For example, any grammar that allows a marked structure like a closed syllable (VC) must also allow the unmarked counterpart (CV).

Figure 2: A linear probability surface typical of Stochastic Harmonic Grammar, where the transition between mappings follows a straight trajectory in the weight space.

The MaxEnt Exception

A striking finding of the research is that Maximum Entropy (MaxEnt) models do not share this stability. Unlike SHG, which uses noise to create variation, MaxEnt uses a different mathematical foundation based on the softmax function. This leads to a different "convex geometry" in the model's decision space.

Figure 5: A curved probability surface characteristic of Maximum Entropy models, illustrating the geometric divergence from linear stochastic frameworks.

Because of this geometric difference, MaxEnt is significantly more restrictive in the universals it preserves. In the basic syllable system, MaxEnt fails to support 8 of the 16 entailments found in the other models. This suggests that MaxEnt is not merely a "noisy" version of Harmonic Grammar but a fundamentally different theoretical animal that may make phonologically implausible predictions regarding language universals.

Conclusion

The stability of T-orders in SOT and SHG provides a strong theoretical justification for using these models to study variation without fearing that the core logic of phonology will be lost. However, the divergence of MaxEnt highlights the importance of the mathematical "shape" of a grammar. Future work remains to fully characterize the computational challenges of calculating these T-orders in larger, more complex constraint sets.

Summary updated

BibTeX
@inproceedings{magri2018implicational,
  author = {Giorgio Magri},
  title = {{Implicational Universals in Stochastic Constraint-Based Phonology}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2018},
  doi = {10.18653/v1/d18-1364},
}
Figure