Equiprobable mappings in weighted constraint grammars

Arto Anttila, Scott Borgeson, Giorgio MagriACL 2019

TL;DR

This paper investigates the concept of "equiprobable mappings" in probabilistic phonological grammars, specifically comparing Maximum Entropy (ME) and Stochastic Harmonic Grammar (SHG). The authors demonstrate that ME grammars are so powerful they can distinguish between virtually any two mappings, meaning equiprobability is impossible. In contrast, SHG allows for equiprobable mappings, which align with categorical phonological equivalence.

What Problem Does This Paper Address?

Phonological systems often exhibit equivalences: certain sounds or structures behave the same way in a given process, regardless of other properties. For example, vowel harmony might apply to all back vowels, ignoring whether they are high or low. Categorical constraint-based phonology (like Harmonic Grammar) captures this naturally. However, extending this to probabilistic models, where grammars assign probabilities to outputs, is challenging. How do we define and identify phonological equivalence in probabilistic grammars? This paper tackles this by defining "equiprobable mappings" – pairs of input-output mappings that always receive the same probability from a given grammar. They ask: which probabilistic frameworks allow for such mappings, and what do they reveal about the frameworks' expressive power?

Methods

The core of the paper lies in a formal comparison of ME and SHG using convex geometry. They leverage existing results (Anttila and Magri, 2018) that relate uniform probability inequalities to geometric properties of "difference vectors." These vectors capture how much each constraint is violated by a potential output compared to the winning output.

  • SHG: Equiprobable mappings in SHG are characterized by their difference vectors being proportional to each other. This means that if one mapping's difference vector can be scaled to match another's, they are equiprobable. This aligns with categorical Harmonic Grammar equivalence.
  • ME: ME, due to a normalization condition on its probability calculations, requires a stricter geometric condition. Equiprobable mappings in ME must have identical sets of difference vectors. This extreme condition means ME can distinguish between almost any two mappings, making equiprobability impossible.

Datasets

The paper uses a test case of Finnish word stress. They analyze a corpus of approximately 9 million Finnish nouns to evaluate the predictions of SHG regarding equiprobable mappings in this domain. No new datasets were introduced.

Key Insights and Ideas

  • ME's Extreme Richness: The most striking insight is that ME's probabilistic machinery is so powerful it can, in principle, assign different probabilities to any two distinct mappings. This means the notion of phonological equivalence (as captured by equiprobability) is fundamentally absent in ME.
  • SHG's Connection to Categorical Grammar: SHG's equiprobability condition mirrors the equivalence found in categorical Harmonic Grammar. This suggests that SHG, while probabilistic, retains a crucial aspect of phonological equivalence that ME loses.
  • Geometric Interpretation: The use of convex geometry provides an elegant and intuitive way to visualize and understand the differences in expressive power between ME and SHG. The "light gray regions" in the figures visually represent the geometric conditions for equiprobability.
  • Finnish Stress as a Diagnostic: The Finnish stress case study provides empirical evidence. SHG's predictions of equiprobability align well with corpus data for most cases, while ME's inability to allow equiprobability leads to counterintuitive predictions and reversals of observed patterns.

Key Results

  • Proposition 1 (ME): Two mappings (x, y) and (x, y) are equiprobable in ME if and only if their corresponding sets of difference vectors coincide. This implies ME has no equiprobable mappings unless the mappings are identical in terms of constraint violations.
  • Proposition 2 (SHG): Two mappings (x, y) and (x, y) are equiprobable in SHG if and only if their non-redundant difference vectors are proportional to each other. This is equivalent to categorical HG equivalence.
  • Empirical Support for SHG: Analysis of Finnish word stress data shows that SHG's predictions of equiprobability are largely supported by corpus frequencies. In cases where SHG seemed to struggle, ME did not offer a better solution and instead produced counterintuitive reversals.

Limitations and Future Work

  • ME's Counterintuitive Predictions: The paper highlights that ME's inability to allow equiprobability leads to counterintuitive predictions in the Finnish stress case, reversing observed patterns of /t/-deletion/retention.
  • Future Work: The authors suggest that the findings provide preliminary empirical support for SHG over ME, particularly concerning the preservation of phonological equivalence. Further investigation into the implications of ME's extreme richness and its potential for overgeneration in linguistic theory is warranted. The authors also note that the formal proofs are omitted for brevity but are available in a longer version of the paper.

Summary updated

BibTeX
@inproceedings{anttila2019equiprobable,
  author = {Arto Anttila and Scott Borgeson and Giorgio Magri},
  title = {{Equiprobable mappings in weighted constraint grammars}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2019},
}
Figure