TL;DR
This paper presents a novel semi-automatic method for classifying words in Nêhiyawêwin (Plains Cree) by leveraging pre-trained English word embeddings. This approach offers a faster, more linguistically grounded alternative to manual classification or relying on existing, often English-centric, ontologies, particularly for under-resourced languages.
What Problem Does This Paper Address?
Semantic classification of words within a part of speech is crucial for various linguistic studies, enabling researchers to group words into cohesive semantic subclasses for use as predictors in predictive models. While extensive semantic ontologies exist for well-resourced languages like English, such resources are scarce for minority languages, including Nêhiyawêwin. Creating these ontologies manually is time-consuming and expensive. Existing automatic methods often rely on English ontologies like WordNet, which may not accurately capture the nuances of a language's own lexicalization process. This paper aims to bridge this gap by developing a semi-automatic method for Nêhiyawêwin that respects its unique linguistic structure.
Methods
The core of the methodology involves leveraging pre-trained English word embeddings (specifically, the Google News Corpus embeddings) to represent Nêhiyawêwin words. The process begins by using a bilingual Nêhiyawêwin-English dictionary (Wolvengrey, 2001) which provides English definitions for Nêhiyawêwin words.
- English Definition Vectorization: For each Nêhiyawêwin word, its English definition is processed. Stop words are removed, and content words are identified. If a content word from the definition is not directly present in the Google News Corpus, synonyms are used or the word is omitted. American English spellings are converted to Canadian English to match the dictionary's source.
- Naive Sentence Vector Creation: Each word in the English definition is associated with its corresponding 300-dimensional Google News vector. These vectors are then averaged, dimension by dimension, to create a single 300-dimensional vector representing the entire English definition. This is termed a "naive sentence vector."
- Hierarchical Agglomerative Clustering (HAC): Euclidean distance matrices are computed from these naive sentence vectors. HAC is then applied using the Ward method to group words with similar definition vectors.
- Semi-Automatic Post-processing: The clusters generated by HAC are then manually reviewed and adjusted. This post-processing step is crucial for refining the semantic coherence of the clusters, especially for verbs, and involves moving items between clusters, splitting classes, or merging them based on semantic intuition.
The authors also pre-categorize Nêhiyawêwin words into established linguistic classes (e.g., intransitive verbs with animate/inanimate arguments, transitive verbs with animate/inanimate goals; animate/inanimate nouns) before applying HAC, which helps to focus the clustering process.
Datasets
- Nêhiyawêwin-English Dictionary: Wolvengrey (2001) "Nêhiyawēwin: itwēwina = Cree: words" served as the primary source for Nêhiyawêwin words and their English definitions. This dictionary contains approximately 21,717 entries.
- English Word Embeddings: Google News Corpus (3 million words, 300-dimensional vectors) was used for representing English words in the definitions.
- Nêhiyawêwin Corpus: A 140,000 token Nêhiyawêwin corpus (Arppe et al., 2020) was used in initial experiments but found to be insufficient for effective word vector generation.
- Ahenakew-Wolfart Corpus: A subset of this corpus was used for the evaluation of the Nêhiyawêwin Order alternation.
Key Insights and Ideas
- Leveraging English Embeddings for Under-Resourced Languages: The core idea of bootstrapping semantic classification for a low-resource language using high-resource language embeddings is a powerful and generalizable approach.
- Endogenous vs. Exogenous Classification: The method prioritizes understanding Nêhiyawêwin's lexicalization process on its own terms, rather than forcing it into pre-existing, potentially ill-fitting, English-centric ontologies.
- "Naive Sentence Vectors" as a Proxy: Averaging word embeddings for definition words, while simple, proved effective enough to generate meaningful clusters, demonstrating the robustness of distributional semantics.
- Semi-Automatic is Key: While HAC provides a strong starting point, manual post-processing is essential for achieving high-quality, semantically coherent clusters, especially for complex word classes like verbs.
- Linguistic Structure Matters: Pre-categorizing words into established Nêhiyawêwin grammatical and semantic classes (e.g., verb types, noun animacy) significantly improves the clustering outcome.
Key Results
- The semi-automatic method successfully produced semantically cohesive clusters for Nêhiyawêwin nouns and verbs.
- Nouns generally required less post-processing than verbs, with HAC producing relatively good clusters out-of-the-box.
- Verbs, particularly VAI and VTA classes, required substantial manual post-processing to refine cluster coherence.
- The resulting clusters were evaluated for their ability to predict a Nêhiyawêwin morphological alternation (Nêhiyawêwin Order). Both manually adjusted and HAC-only clusters showed a non-negligible ability to explain this phenomenon, with manually adjusted classes performing slightly better for some verb classes.
- The pseudo-R² values for predicting the alternation ranged from 0.01 to 0.19, indicating that the semantic classes derived from this method have tangible linguistic utility.
Limitations and Future Work
- Corpus Size: The initial attempts to generate Nêhiyawêwin word vectors directly from a limited corpus were unsuccessful, highlighting the dependence on external, larger corpora (like English WordNet).
- "Naive Sentence Vectors": The authors acknowledge that more sophisticated methods for sentence vector representation (e.g., phrase vectors) could be explored to better handle non-compositional idioms.
- Manual Post-processing Effort: For verbs, the manual post-processing effort was significant, approaching the time required for full manual classification.
- Evaluation Method: The authors suggest that further research should investigate alternative evaluation methods, such as using speaker input or visual methods, to assess cluster quality.
- Generalizability: While the technique is presented as applicable to other languages, further work is needed to explore its effectiveness across different linguistic typologies and dictionary structures.
- Weighting Definition Words: Investigating methods to weight English words within definitions (e.g., using tf-idf) could potentially improve the semantic representation.
Summary updated