Using Graph-Based Methods to Augment Online Dictionaries of Endangered Languages

Khalid Alnajjar, Mika Hämäläinen, Niko Tapio Partanen, Jack RueterACL 2022

TL;DR

This paper tackles the challenge of creating comprehensive online dictionaries for endangered languages by using graph-based methods to predict new translations. By leveraging existing multilingual dictionaries and Wiktionaries, they successfully generated a significant number of accurate translations, with a neural network achieving 81% accuracy in identifying good predictions.

What Problem Does This Paper Address?

Endangered languages often suffer from a lack of comprehensive digital resources, particularly dictionaries. While many such languages have existing bilingual dictionaries, these are frequently in print, limiting their accessibility in the digital age. Furthermore, existing digital dictionaries are often inconsistent in their language pairings and may not cover all necessary translations. This paper aims to bridge this gap by automatically augmenting these dictionaries with new translations, making them more useful for speakers and language learners.

Methods

The core of the paper's methodology involves representing translation data as graphs. Words are nodes, and translations are edges. They then employ four link prediction algorithms (Jaccard coefficient, Adamic-Adar index, resource allocation index, and preferential attachment score) to infer new translation links between words in different languages. These algorithms analyze the connectivity and common neighbors within the graph to suggest potential translations. To automate the quality assessment of these predictions, they train a multilayer feedforward neural network. This network takes the scores from the link prediction methods as input and outputs a binary prediction of whether a translation is "good" or "bad."

Datasets

The study utilizes two main types of data:

  • XML dictionaries of endangered languages: Specifically, dictionaries for Komi-Zyrian (kpv), Erzya (myv), and Livonian (liv). These dictionaries were created through collaborative development efforts.
  • Wiktionaries: Publicly available Wiktionaries for resource-rich languages such as Finnish, Estonian, French, Latvian, and Russian.

The authors also created a manually annotated dataset of 800 predicted translations for evaluation purposes, categorizing them as "good," "acceptable," "incomplete," or "bad."

Key Insights and Ideas

  • Graph-based inference is powerful for low-resource settings: Representing translations as graphs allows for the effective utilization of incomplete but complementary lexical resources, a crucial advantage when dealing with data-scarce endangered languages.
  • Leveraging high-resource languages is key: By using Wiktionaries of well-resourced languages, the method can infer translations for endangered languages, effectively "bootstrapping" new entries into existing dictionaries.
  • Automated quality assessment is feasible: Training a neural network to predict the quality of automatically generated translations significantly reduces the manual effort required for verification.
  • Manual evaluation remains critical: Despite promising automated results, human evaluation by fluent speakers is essential for ensuring the accuracy and utility of the generated translations, especially given the nuances of polysemy and potential errors in source dictionaries.

Key Results

  • The graph-based prediction method achieved 77% good or acceptable translations when evaluated manually by fluent speakers.
  • A neural network model trained to predict translation quality achieved 81% accuracy.
  • The predicted translations are made available on the Akusanat online dictionary platform, directly benefiting language communities.

Limitations and Future Work

  • Part-of-speech information: The current graph representation omits part-of-speech tags, which could lead to ambiguous predictions. The authors suggest incorporating automatically predicted POS tags as a future step.
  • Polysemy and source dictionary errors: The method can be misled by words with multiple meanings (polysemy) or inaccuracies in the original dictionaries, leading to incorrect predictions.
  • Synthetic data generation challenges: Creating realistic synthetic datasets for training the neural network, especially for "bad" translations, proved difficult due to the lack of representative examples.
  • Future work: The authors envision integrating their method into existing dictionary editing infrastructures for a more dynamic, human-in-the-loop approach. They also suggest exploring the construction of aligned multilingual word embeddings for endangered languages.

Summary updated

BibTeX
@inproceedings{alnajjar2022using,
  author = {Khalid Alnajjar and Mika Hämäläinen and Niko Tapio Partanen and Jack Rueter},
  title = {{Using Graph-Based Methods to Augment Online Dictionaries of Endangered Languages}},
  booktitle = {Annual Meeting of the Association for Computational Linguistics},
  year = {2022},
}
Figure