TL;DR
This paper proposes a novel information-theoretic model that predicts adjective order across diverse languages by maximizing "information gain." This approach successfully explains adjective ordering patterns in pre-nominal, post-nominal, and mixed-order languages, offering a unified explanation for asymmetries that previously required language-specific rules.
What Problem Does This Paper Address?
Languages exhibit fascinating, yet often complex, rules for ordering multiple adjectives before a noun. For instance, English speakers consistently prefer "big blue box" over "blue big box." While some languages, like English (AAN) and Arabic (NAA), have consistent pre- or post-nominal adjective ordering, others, like French (ANA), mix pre- and post-nominal positions. Traditional linguistic theories often rely on hierarchical syntactic structures or semantic properties like "inherentness" or "subjectivity" to explain these preferences. However, these approaches struggle to account for the mixed-order languages without ad-hoc additions and don't always provide a unified explanation across different language types. This paper aims to bridge this gap by proposing a single, quantitative, information-theoretic principle that can predict adjective order across all these typological templates.
Methods
The core of the paper's methodology lies in framing adjective ordering as a decision tree problem. The authors propose that a speaker is trying to convey a meaning (represented as a binary feature vector) to a listener who has an initial probability distribution over possible meanings. Each word in a phrase, particularly adjectives, reduces the listener's uncertainty.
The key innovation is the use of information gain (IG), a measure from machine learning that quantifies the reduction in uncertainty. IG is calculated by considering both "positive evidence" (how much a word's presence reduces uncertainty) and "negative evidence" (how much the absence of a word reduces uncertainty). This dual consideration is crucial because it breaks the symmetry inherent in purely positive evidence-based measures (like surprisal), allowing for the prediction of left-right asymmetries observed in adjective order.
Specifically, they train a logistic regression model where the independent variable is the difference in information gain between two possible adjective orders, and the dependent variable is whether that order is attested in the corpus. This allows them to assess how well IG predicts observed orders and to quantify the strength of this prediction.
Datasets
The study utilizes data extracted from the CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies. This includes:
- Wikipedia dumps: Used to extract noun phrases (NPs) with adjectives to build feature vectors and their probability distributions.
- Common Crawl dumps: Used to extract sequential triples of (noun, adjective, adjective) for training and hold-out accuracy testing.
The authors focus on languages with at least 5,000 triples and templates (AAN, ANA, NAA) representing at least 10% of a language's triples.
Key Insights and Ideas
- Information Gain as a Unified Predictor: The most significant insight is that maximizing information gain, by considering both positive and negative evidence, provides a single, powerful principle for predicting adjective order across diverse typological templates (AAN, ANA, NAA).
- Breaking Symmetry with Negative Evidence: The inclusion of "negative evidence" in the information gain calculation is crucial. It allows the model to account for asymmetries in adjective order, particularly in mixed-order languages (ANA), which have been challenging for previous models.
- Adjective Order as a Decision Tree: Conceptualizing the noun phrase as a decision tree where each adjective "partitions" the space of possible meanings offers an intuitive and powerful framework for understanding ordering preferences.
- Robustness Across Languages: The model demonstrates consistent predictive power across a wide range of languages, suggesting that the underlying principle of information gain is a fundamental driver of adjective ordering.
Key Results
- High Predictive Accuracy: The model achieves a comprehensive accuracy of 68.7% across all languages and templates.
- AAN triples: 65.6% accuracy
- ANA triples: 73.7% accuracy
- NAA triples: 68.0% accuracy
- Strong Predictor: The regression coefficient ($\beta_1$) is positive in 93.2% of the tested datasets, indicating a strong preference for placing the adjective with higher information gain first.
- Cross-Linguistic Consistency: The model's performance is remarkably consistent across different language families and adjective ordering templates, with overlapping confidence intervals for $\beta_1$ across templates.
- Comparison to Other Metrics: The proposed IG-based approach outperforms other metrics like subjectivity, PMI, and integration cost in predicting English adjective order, as reported in previous work.
Limitations and Future Work
- Data Limitations: The study is limited to languages with sufficient data (at least 5k triples) and specific templates. The authors acknowledge that some languages might have fewer triples for certain templates, potentially affecting results.
- Focus on Triples: The analysis primarily focuses on noun-adjective-adjective triples to minimize the influence of other dependents. Future work could explore more complex noun phrases.
- "Many" as an Adjective: The paper notes a potential ambiguity in classifying words like "many" as adjectives, which can vary across languages and annotation schemes.
- Future Directions: The authors suggest that their IG-based approach could be extended to other areas of syntax and semantics where ordering preferences are observed. Further exploration into the specific functional pressures that lead to certain IG values could also be fruitful.
Summary updated