TL;DR
This paper proposes a novel measure of "integration complexity" based on the entropy of a word's syntactic head categories. They demonstrate that this complexity, alongside dependency distance, can explain a widespread cross-linguistic preference for ordering "cosisters" (syntactically similar dependents on the same side of their head) such that the one with lower integration complexity is placed closer to the head.
What Problem Does This Paper Address?
Linguists have long observed that certain word orders are preferred over their reverses, even when they are semantically equivalent. For instance, "pretty blue fish" sounds more natural than "blue pretty fish." While previous models have proposed specific hierarchies for different word types (like adjectives or prepositional phrases), these models often lack a unifying principle and external motivation. This paper aims to provide a more general theory of constituent ordering by addressing the "integration cost" – the cognitive effort involved in processing dependent words and their heads. Specifically, it tackles the ordering of "isomorphic cosisters," which are syntactically similar constituents that appear together, like stacked adjectives or multiple prepositional phrases.
Methods
The core of the paper's contribution is the introduction of Integration Complexity (IC). This is calculated by:
- Identifying the syntactic categories of a word's heads: For a given dependent word, the researchers find all the words it depends on (its heads) in a corpus and determine their part-of-speech tags.
- Calculating the probability distribution of these head categories: They then compute the proportion of times the dependent word attaches to each syntactic category.
- Computing the entropy of this distribution: Using Shannon entropy (H(X) = - Σ p(x) * log₂(p(x))), they quantify the "peakedness" of this distribution. A low entropy means the dependent word consistently attaches to a few specific head categories (making it predictable and thus less complex to integrate), while a high entropy indicates a wide variety of head categories, making it more complex.
This IC measure is then combined with Dependency Distance Minimization (DDM), a previously established principle that favors shorter distances between dependents and their heads. The hypothesis is that the integration cost (sum of a dependent's IC and the distance to its head) influences word order. Specifically, for isomorphic cosisters, the one with lower IC is predicted to be placed closer to the head.
Datasets
The study utilizes the Universal Dependencies (UD) corpora. The authors specifically mention using the English-EWT (English Web Treebank) corpus for detailed examples and analysis. They also conducted a large-scale analysis across 70 languages available in the UD project to test their hypothesis. The UD corpora are valuable because they provide consistent, cross-linguistically annotated dependency parses, allowing for the calculation of head categories and distances.
Key Insights and Ideas
- Unifying Principle: Integration complexity, derived from the entropy of head categories, offers a more general and externally motivated explanation for constituent ordering than previous type-specific hierarchies.
- Cosister Ordering: The paper provides strong evidence that isomorphic cosisters are ordered based on their relative integration complexity, with lower complexity constituents tending to appear closer to their heads.
- Subsuming Previous Models: Many existing models for ordering specific types of constituents (e.g., adjectives, noun modifiers) can be explained as epiphenomena of this broader integration complexity principle. For example, the perceived hierarchy of adjectives might arise from their similar integration complexities.
- Iconicity and Efficiency: The preference for lower integration complexity constituents closer to the head aligns with principles of iconicity (syntactic proximity mirroring semantic closeness) and cognitive efficiency.
Key Results
- A large-scale analysis across 70 languages found that 61 out of 70 languages (87.1%) showed a statistically significant pattern where an increasing difference in integration complexity between cosisters led to a higher probability of the lower-complexity cosister being placed closer to the head.
- Examples from English (e.g., "beautiful small black purse," adverb placement in "Sam has probably been called") illustrate how integration complexity predicts preferred orders.
- The study suggests that phenomena like "heavy noun phrase shift" can also be understood as driven by integration complexity, where longer or more complex phrases are moved further from the head.
Limitations and Future Work
- Nonce Words: The paper notes that integration complexity based on mutual information (a related concept) would struggle with nonce words, as they have no frequency. Their proposed entropy-based IC, however, can handle nonce words by relying on the syntactic categories of their heads.
- Multi-word Cosisters: The paper acknowledges that determining the best way to calculate integration complexity for multi-word cosisters is not entirely clear, exploring two methods (summing node ICs vs. treating them as a single unit for DDM) and opting for the simpler summation for the study.
- Language-Specific Noise: The authors suggest that noise from language-specific tagging and lemmatization in UD corpora might contribute to the few languages that don't perfectly conform to the hypothesized pattern.
- Future Work: The paper lays the groundwork for a more inclusive theory of constituent ordering based on integration cost, suggesting that this approach can subsume various ordering phenomena and potentially explain deviations from canonical orders.
Summary updated