Investigating Information-Theoretic Properties of the Typology of Spatial Demonstratives

Sihan Chen, Richard Futrell, Kyle MahowaldNAACL 2022

TL;DR

This paper explores why languages have different ways of expressing spatial relationships like "here" and "there." Using information theory, they show that real-world language systems are surprisingly efficient, balancing how much information they convey with how complex they are. This suggests that language evolution might favor systems that are "just right" in terms of information and complexity.

What Problem Does This Paper Address?

Languages around the world have vastly different systems for spatial demonstratives – words like "here," "there," and "from there." These systems vary in how many "levels" of distance they distinguish (e.g., English has "here," "there," and sometimes "(over) there") and how they combine spatial concepts like location (PLACE), direction of movement towards a point (GOAL), and direction of movement away from a point (SOURCE). For instance, English uses "there" for both PLACE and GOAL but has a distinct word for SOURCE ("from there"). Finnish, on the other hand, has unique words for each combination of distance and orientation.

The core problem is understanding the principles that drive this linguistic diversity. Why do some languages have simple systems while others are more complex? This paper tackles this by asking: can information theory explain the observed patterns in spatial demonstrative systems, suggesting that these systems are shaped by an evolutionary pressure towards efficiency?

Methods

The authors employ an Information Bottleneck (IB) framework. This approach models how information is compressed and transmitted. In this context:

  • World State (U): Represents the actual spatial reality (e.g., a referent is "here" and the speaker is moving "towards" it - GOAL).
  • Meaning (M): The speaker's mental representation of that spatial reality.
  • Words (W): The actual linguistic forms used to express the meaning.

The IB framework quantifies two key properties:

  1. Informativity: The mutual information between the words (W) and the world states (U). This measures how much information a word conveys about the actual spatial situation.
  2. Complexity: The mutual information between the words (W) and the mental representations of meaning (M). This measures how much information is needed to encode the meaning into words.

The paper proposes that efficient language systems optimize a trade-off between these two, aiming to maximize informativity while minimizing complexity. This is mathematically represented by an equation (Eq. 1) that balances these two terms. They also incorporate a "cost function" (Eq. 2) that penalizes confusing different spatial distinctions (e.g., confusing PLACE with GOAL is less costly than confusing PLACE with SOURCE).

Datasets

The study uses data from Nintemann et al. (2020), which comprises spatial deictic lexicons from 223 languages. This dataset serves as the basis for evaluating their information-theoretic model against real-world linguistic systems. The paper does not introduce a new benchmark dataset but rather analyzes an existing, comprehensive one.

Key Insights and Ideas

  • Efficient Frontier: Real-world spatial demonstrative lexicons tend to cluster along an "efficient frontier." This means they are not arbitrarily complex or simple but rather represent a near-optimal balance between conveying information and minimizing linguistic effort.
  • Cost Function Matters: The specific "cost" assigned to confusing different spatial distinctions is crucial. The authors find that a plausible cost function, where confusing SOURCE with other orientations is more costly than confusing PLACE and GOAL, helps explain why real lexicons lie on the efficient frontier.
  • Prior Probabilities Influence Design: The "prior probability" of certain meanings also plays a role. The finding that SOURCE orientations might have lower prior probability (meaning they are less frequently needed or distinguished) can explain their specific treatment in some languages.
  • Systematicity is Key: Beyond just efficiency, real lexicons exhibit "systematicity." This means the patterns of distinctions are consistent across different distance levels (e.g., if a language distinguishes PLACE and GOAL at "here," it should do so similarly at "there"). This adds another layer of explanation beyond pure information-theoretic efficiency.

Key Results

  • The paper demonstrates that real-world spatial demonstrative lexicons from 223 languages fall close to an information-theoretic efficient frontier, supporting the idea that language evolution favors efficiency.
  • They identify specific properties of the cost function and prior probabilities that are necessary for real lexicons to align with this efficient frontier. Notably, the cost of confusing SOURCE with other orientations is higher, and SOURCE orientations may have lower prior probability.
  • The authors show that while information-theoretic efficiency is important, it doesn't fully explain all attested patterns. The concept of "systematicity" is introduced to account for the consistent structural patterns observed across different distance levels in real lexicons.

Limitations and Future Work

  • IB Model Limitations: The authors acknowledge that the IB approach, while powerful, doesn't fully capture all observed patterns in human lexicons. Specifically, there exist theoretically efficient lexicons that are not attested in real languages, suggesting other factors (like systematicity) are also at play.
  • Future Work: The paper suggests that the methodological innovations developed here could be applied to information-theoretic analyses of other typological domains in linguistics. This opens up avenues for exploring efficiency principles in a wider range of linguistic phenomena. The concept of systematicity also warrants further investigation as a distinct but complementary principle shaping language.

Summary updated

BibTeX
@inproceedings{chen2022investigating,
  author = {Sihan Chen and Richard Futrell and Kyle Mahowald},
  title = {{Investigating Information-Theoretic Properties of the Typology of Spatial Demonstratives}},
  booktitle = {Conference of the North American Chapter of the Association for Computational Linguistics},
  year = {2022},
}
Figure