A Computational Model for the Assessment of Mutual Intelligibility Among Closely Related Languages

Jessica Nieder, Johann-Mattis ListEACL 2024

TL;DR

This paper introduces a computational model to automatically assess how well speakers of closely related languages can understand each other. By leveraging linguistic features like phonetic sound classes and semantic word embeddings, their model shows promising results that mirror human intelligibility patterns, offering a scalable alternative to traditional psycholinguistic experiments.

What Problem Does This Paper Address?

Mutual intelligibility, the ability of speakers of one language to understand speakers of another without explicit learning, is a crucial concept in linguistics. Traditionally, assessing this involves time-consuming and resource-intensive psycholinguistic experiments with human participants. These studies face challenges like finding participants with truly zero prior exposure to the target language, ensuring methodological uniformity, and scaling to multiple languages. This paper aims to bridge this gap by developing a computational model that can predict mutual intelligibility, thereby overcoming the limitations of human-based studies and offering a more objective and scalable assessment tool.

Methods

The core of the proposed method is the Linear Discriminative Learner (LDL), a computational model that simulates language comprehension by mapping phonological forms to semantic representations. The authors extend this framework by incorporating:

  • Multilingual Semantic Vectors: They utilize pre-trained multilingual word embeddings from ConceptNet Numberbatch. These embeddings allow for direct comparison of the meanings of words across languages.
  • Multilingual Sound Classes: Instead of raw phonetic transcriptions, they employ a broader representation using Dolgopolsky's sound classes (with an added symbol for vowels). This approach captures more general phonetic similarities, crucial for cross-linguistic comparisons, and is inspired by computational methods in historical linguistics.
  • Trimming Word Forms: To focus on core lexical meaning and reduce the influence of grammatical endings, they apply a technique to automatically trim word forms, effectively creating "bare stems." This is achieved by removing phonetic alignment columns with a high proportion of gaps, indicating sounds not consistently present across languages.

The LDL model is trained on cognate sets from German, Dutch, and English. Comprehension is assessed by predicting the semantic vector of a word form and comparing it to the gold standard semantic vector.

Datasets

The primary dataset used is a collection of 340 cognate sets derived from Kluge's etymological dictionary. These cognates have reflexes in German, Dutch, and English, with proto-forms in Proto-Germanic. The dataset includes phonetic transcriptions and phonetic alignments. For semantic vectors, they used ConceptNet Numberbatch word embeddings (version 19.08). One cognate was removed due to missing Dutch embeddings, resulting in a final dataset of 339 cognates.

Key Insights and Ideas

  • Computational Mimicry of Human Intelligibility: The model's ability to replicate human intelligibility patterns, particularly regarding the influence of language pairs and the impact of trimming word forms, is a significant finding. This suggests that the chosen linguistic features (sound classes and semantic embeddings) are indeed key drivers of mutual intelligibility.
  • The Power of Sound Classes: Using broader phonetic representations (sound classes) rather than fine-grained IPA transcriptions proves effective for capturing cross-linguistic phonetic similarities relevant to intelligibility.
  • Trimming for Deeper Understanding: Automatically trimming word forms to bare stems significantly improves comprehension accuracy, highlighting the role of morphological endings in obscuring core lexical similarity and suggesting that speakers might implicitly "ignore" such endings when assessing intelligibility.
  • English as a "Least Advantageous" Training Language: Similar to human studies, training the model on English data yields lower comprehension accuracies, suggesting that English might be more divergent or that its cognates have undergone more significant changes compared to German and Dutch.

Key Results

  • High Accuracy with Trimmed Forms: After trimming word forms, comprehension accuracies for the multilingual models significantly increased. For example, the German-Dutch pair achieved accuracies of 0.81 (4-grams) and 0.75 (3-grams) for top-1 candidate prediction.
  • 4-gram and 3-gram Sound Classes are Predictive: The model demonstrates good comprehension accuracy when using 4-gram or 3-gram chunks of sound classes as input. A substantial drop in accuracy is observed with 2-gram chunks, indicating reduced discriminative power.
  • Human-like Performance: The cross-linguistic comprehension results show similarities to human intelligibility data reported in previous studies, particularly in the relative performance of different language pairs and the effect of trimming. For instance, the German-English pair showed better results than German-Dutch in the untrimmed condition, aligning with human data.

Limitations and Future Work

  • Limited Dataset Size: The authors acknowledge that the provided dataset contains a limited number of carefully selected historical cognates. They suggest that future work should explore how the model performs with larger, more random word sets.
  • Language Family Specificity: The current model is tested only on Germanic languages (German, Dutch, English). The authors propose extending the model to other language families to assess its generalizability.
  • Broader Linguistic Factors: While the model focuses on phonetic and semantic similarity, future work could incorporate other linguistic factors that influence intelligibility, such as syntactic structures or orthographic similarity.
  • Asymmetric Intelligibility: The paper notes that their approach, like many others, doesn't explicitly test for asymmetric intelligibility (where speaker A understands speaker B better than vice versa). This remains an area for further investigation.

Summary updated

BibTeX
@inproceedings{nieder2024computational,
  author = {Jessica Nieder and Johann-Mattis List},
  title = {{A Computational Model for the Assessment of Mutual Intelligibility Among Closely Related Languages}},
  booktitle = {EACL},
  year = {2024},
}
Figure