TL;DR
This paper presents a novel method for building a wordnet for Abui, a low-resource language, by leveraging existing Toolbox dictionaries and multilingual sense intersection. This approach successfully creates a high-quality wordnet with a high accuracy rate, demonstrating a viable path for developing lexical resources for under-represented languages.
What Problem Does This Paper Address?
Developing comprehensive lexical resources like wordnets for low-resource languages is a significant challenge. Traditional methods are often slow, labor-intensive, and may not adequately capture the unique semantic nuances of these languages, especially when they differ substantially from high-resource languages like English. Existing dictionaries for these languages, while valuable, often have gaps and are not structured for easy integration into computational linguistic frameworks. This paper addresses the need for an efficient and effective method to bridge this gap, enabling the creation of wordnets for languages with limited digital linguistic data.
Methods
The authors propose a bootstrapping technique that combines data from a Toolbox dictionary of the Abui language with existing wordnets for English, Indonesian, and Malay. The core of their method is multilingual sense intersection. This involves:
- Extracting Data: Information from the Abui Toolbox dictionary, including lemmas, part-of-speech, reverse glosses, and definitions, is extracted.
- Sense Candidate Generation: A naive algorithm is used to generate potential senses for Abui words by finding overlaps (intersections) in the senses of equivalent words in the English, Indonesian, and Malay wordnets. This leverages the idea that if a word has similar meanings across multiple languages, those meanings are more likely to be correct for the target language.
- Ranking Candidates: Candidate senses are ranked based on the number of languages that suggest them and the degree of overlap with definitions.
- Manual Verification: The generated candidates are then manually checked and refined.
This approach is an instance of the "expansion approach" to wordnet building, using existing wordnets as pivots.
Datasets
- Abui Toolbox Dictionary: This is the primary source of Abui lexical data, containing lemmas, pronunciations, parts-of-speech, and multilingual glosses (English, Indonesian, Malay).
- Princeton Wordnet: Used as a pivot for English.
- Wordnet Bahasa: Used as a pivot for Indonesian and Malay.
- Extended Open Multilingual Wordnet (OMW): Data from OMW was also utilized, which includes automatically collected data from Wiktionary and CLDR, as well as ongoing sense annotation efforts.
The paper introduces a newly developed Abui wordnet (v1.0) as a result of this process, containing over 1,400 concepts and 3,600 senses.
Key Insights and Ideas
- Multilingual Sense Intersection is Powerful: The core insight is that aligning senses across multiple related languages (even if not directly related to the target language) can significantly improve the accuracy of identifying correct word senses for a low-resource language.
- Triangulation is Key: Senses supported by three languages (English, Indonesian, Malay) showed exceptionally high accuracy (99%), highlighting the benefit of "triangulation" for robust sense identification.
- Toolbox Dictionaries are Rich Resources: The SIL Toolbox format, with its detailed fields for glosses, definitions, and examples, provides a wealth of information that can be effectively leveraged for wordnet construction.
- Naive Algorithms Can Be Effective: The paper demonstrates that even a simple algorithm for sense intersection and ranking can yield high-quality results when combined with manual verification, making it accessible for linguists without extensive computational linguistics backgrounds.
- Open Data is Crucial: The authors advocate for releasing linguistic data under open licenses to foster collaboration and accelerate the development of resources for low-resource languages.
Key Results
- The developed Abui wordnet (v1.0) contains 1,475 synsets and 3,606 senses.
- Three-way intersection (Abui, English, Indonesian, Malay) yielded candidate senses with an accuracy of 99%.
- Two-way intersection achieved an accuracy of 50%.
- Single-language intersection had an accuracy of 35%.
- The ranking algorithm, while naive, was effective in differentiating candidates, with higher-scoring candidates showing better accuracy.
Limitations and Future Work
- Algorithm Simplicity: The authors acknowledge that their sense intersection algorithm is naive and could be improved by incorporating more sophisticated features like semantic domain information, wordnet hierarchies, and detailed glosses/definitions.
- Coverage: The current wordnet has a lower coverage of adjectives and adverbs, which is attributed to the Abui language's tendency to express event properties using stative verbs.
- Future Work: The authors plan to:
- Incorporate pronunciation, grammatical information (aspectual class, valency, etymology), and example sentences from the Toolbox dictionary.
- Address Abui-specific concepts, regional taxonomies (animals, plants), and non-lexicalized CILI concepts.
- Handle the lack of an official Abui orthography by aggregating and linking alternative spellings.
- Continue to improve the sense intersection algorithm.
Summary updated