Automatic Discovery of Adposition Typology

Rishiraj Saha Roy, Rahul Katare, Niloy Ganguly, Monojit ChoudhuryCOLING 2014

TL;DR

This paper introduces an unsupervised method to automatically classify languages as prepositional or postpositional based on word co-occurrence statistics. This approach is crucial for understanding linguistic typology, especially for under-resourced languages, and has implications for NLP tasks like query understanding.

What Problem Does This Paper Address?

Languages can be broadly categorized by how they use adpositions (prepositions, postpositions, etc.) in relation to noun phrases. For example, English uses prepositions ("in the house"), while Japanese uses postpositions ("house in"). This "adposition typology" is correlated with other linguistic features and is valuable for linguistic analysis and NLP. However, manually determining this typology for many languages, especially those with limited digital text, is time-consuming and expensive. While automatic methods exist for some typological features, the automatic discovery of adposition typology has received little attention. This paper aims to fill this gap by proposing a computationally efficient, unsupervised method to predict a language's adposition typology.

Methods

The core idea is to leverage the distinct distributional properties of function words (like adpositions) compared to content words. The authors make two key assumptions:

  1. Adpositions are function words: A significant portion of function words are adpositions.
  2. Distributional differences: The strict ordering of adpositions with noun phrases leads to different co-occurrence patterns on their left and right sides compared to content words.

The method involves:

  1. Function Word Detection: They hypothesize that function words co-occur with a larger variety of other words and exhibit less biased co-occurrence patterns than content words. They measure this using:
  2. Co-occurrence Counts (LCC, RCC, TCC): The number of distinct words appearing immediately to the left (LCC), right (RCC), or either side (TCC) of a target word.
  3. Co-occurrence Entropies (LCE, RCE, TCE): The entropy of the probability distribution of words appearing to the left (LCE), right (RCE), or either side (TCE) of a target word. Higher entropy suggests more unpredictable co-occurrences.
  4. Adposition Typology Prediction: Based on the observation that LCE/LCC are better indicators for prepositional languages and RCE/RCC for postpositional languages, they propose a method using rank correlations. They generate ranked lists of words based on TCE, LCE, and RCE (and their count counterparts). For a given language, they compute the Spearman's rank correlation coefficient between the list sorted by TCE and the list sorted by LCE (ρ(TL)), and between TCE and RCE (ρ(TR)).
  5. If ρ(TL) > ρ(TR), the language is predicted as prepositional.
  6. If ρ(TL) < ρ(TR), the language is predicted as postpositional.

Datasets

The experiments were conducted on corpora from the Leipzig Corpora collection.

  • Function Word Detection: Experiments were performed on four languages: English, Italian, Hindi, and Bangla.
  • Adposition Typology Detection: Experiments were conducted on 23 languages from ten diverse families. These languages and their true typologies are listed in Table 3.

Key Insights and Ideas

  • Function words have distinct distributional profiles: The paper convincingly shows that function words, particularly adpositions, can be identified by their tendency to co-occur with a wider range of words and exhibit less skewed co-occurrence distributions compared to content words.
  • Left vs. Right context matters for typology: The key insight is that the predictive power of left-context statistics (LCC, LCE) is stronger for prepositional languages, while right-context statistics (RCC, RCE) are stronger for postpositional languages. This asymmetry is directly linked to the syntactic position of adpositions.
  • Entropy is a robust indicator: Co-occurrence entropy measures (LCE, RCE, TCE) generally perform better than raw counts (LCC, RCC, TCC) for function word detection.
  • TCE/TCC as a proxy for function words: The "total" co-occurrence statistics (TCE, TCC) are shown to be good, general predictors of function words, performing reasonably well across both prepositional and postpositional languages. This makes them useful for initial ranking when the typology is unknown.
  • Rank correlation as a typology predictor: The clever use of Spearman's rank correlation between entropy-based lists (TCE vs. LCE/RCE) provides a robust and simple mechanism to distinguish between prepositional and postpositional languages.

Key Results

  • The proposed method correctly predicts the adposition typology for 19 out of 23 languages tested.
  • For function word detection, co-occurrence statistics consistently outperform simple frequency as indicators, especially when evaluated using Average Precision (AP@200).
  • The method is robust to variations in corpus size (down to 10k sentences) and vocabulary size, though performance is best with larger corpora for initial ranking.
  • The authors note that highly synthetic languages (e.g., from the Uralic family) were misclassified, likely due to extensive agglutination and case-marking that obscure the role of adpositions as free words.

Limitations and Future Work

  • Synthetic Languages: The method struggles with highly synthetic and inflectional languages where case markers or affixes might play a role similar to adpositions, making it difficult to isolate the adposition's distributional signature.
  • Ambiguity: For languages where the difference in correlation coefficients is very small (e.g., Lithuanian), the prediction is inconclusive.
  • Future Work:
    • Improve prediction accuracy by including more languages and refining corpus size/significance thresholds.
  • Extend the approach to a morpheme-level analysis to better handle highly agglutinative languages.
  • Investigate the application of this technique to emergent linguistic systems (pidgins, CMCs) and undeciphered scripts.

Summary updated

BibTeX
@inproceedings{roy2014automatic,
  author = {Rishiraj Saha Roy and Rahul Katare and Niloy Ganguly and Monojit Choudhury},
  title = {{Automatic Discovery of Adposition Typology}},
  booktitle = {International Conference on Computational Linguistics},
  year = {2014},
}
Figure