TL;DR
This paper presents a novel approach to modeling the complex morphology of Lushootseed, an endangered language, by combining a Finite State Transducer (FST) with a Long Short-Term Memory (LSTM) neural network. This hybrid method successfully generates training data for the LSTM, achieving a 71.9% accuracy in morphological analysis, and offers a robust framework for handling the language's intricate reduplication patterns.
What Problem Does This Paper Address?
Many endangered languages, like Lushootseed, suffer from a scarcity of readily available textual data, posing a significant challenge for Natural Language Processing (NLP) research. This lack of data hinders the development of computational tools for tasks such as morphological analysis, lemmatization, and generation. Furthermore, Lushootseed exhibits complex reduplication patterns, where parts of a word are repeated to convey grammatical or semantic information (e.g., distributive or diminutive meanings). Modeling these patterns accurately is crucial for understanding and preserving the language, but traditional rule-based systems can become unwieldy, and purely data-driven approaches are hampered by the limited data.
Methods
The authors propose a two-pronged approach:
- Finite State Transducer (FST) for Rule-Based Modeling: They develop an FST to capture the phonological and morphological rules of Lushootseed, particularly focusing on its reduplication patterns. A key innovation is the use of a five-place holder system preceding the word root. This system, numbered {p5} to {p1}, allows for the systematic representation of different types of reduplication (distributive, diminutive, and their combinations) by mapping specific phonemes to these placeholders. This FST is designed for machine readability and facilitates supersegmental concatenation and tagging.
- LSTM for Neural Extension and Data Generation: Recognizing the limitations of FSTs in covering an entire lexicon, the paper integrates an LSTM-based neural model. The FST is used to generate a large synthetic dataset of inflectional forms and their corresponding morphological readings for Lushootseed nouns. This generated data is then used to train the LSTM model for morphological analysis. The LSTM acts as a "fallback" mechanism for words not covered by the FST and allows for analysis of out-of-vocabulary words.
Datasets
The primary data sources for this research include:
- Lushootseed dictionary by Bates et al. (1994)
- Language learning binders by Zalmai Zahir and Peggy ki?alq Ahvakana.
- Transcriptions of recordings from the University of Washington archives.
A significant contribution is the creation of a synthetic dataset for training the LSTM. This dataset consists of approximately 756,000 inflectional form-morphological reading tuples derived from 214 Lushootseed nouns using the FST. This dataset was split into 70% training, 15% validation, and 15% testing, with the test data containing words unseen by the model during training.
Key Insights and Ideas
- Hybrid FST-LSTM Approach: Combining rule-based FSTs with data-driven LSTMs offers a powerful solution for low-resource languages. The FST provides linguistic rigor and systematic rule capture, while the LSTM leverages generated data to achieve broader coverage and handle unseen words.
- Five-Place Holder System for Reduplication: The proposed five-place holder system ({p5} to {p1}) is an elegant way to systematically model the complex and varied reduplication patterns in Lushootseed, including combinations of distributive and diminutive forms.
- Synthetic Data Generation: Using an FST to generate large amounts of training data for an LSTM is a viable strategy for overcoming data scarcity in endangered languages. This allows for the development of robust neural models even without extensive human-annotated corpora.
- Handling Orthographic Variation: The paper acknowledges and addresses challenges related to non-standard orthography and character encoding in Lushootseed materials, which is a common issue with digitized historical texts.
Key Results
- The FST-based approach provides a finite-state description of Lushootseed reduplication, handling issues of orthography, morphophonology, and concatenation.
- The LSTM model, trained on synthetically generated data, achieved an overall accuracy of 71.9% on the test corpus for morphological analysis.
- The paper provides per-tag results for the neural model, showing varying levels of precision and recall for different morphological tags.
Limitations and Future Work
- Limited FST Coverage: The FST does not yet cover the entire Lushootseed vocabulary, meaning it cannot process all domains of text. Writing FSTs is also time-consuming and requires specialized knowledge.
- Neural Model Noun Limitation: The current neural model is limited to analyzing nouns only. While it can handle out-of-vocabulary words, its accuracy has only been tested on words known to the FST.
- Derivational Morphology Untrained: The neural model was not trained on derivational morphology, which could lead to erroneous predictions for word derivations.
- Future Work: The authors suggest the need for more test data and descriptions of regular inflection to further test the accuracy of their Lushootseed description. They also propose exploring whether a six-placeholder segment is necessary and aligning their description with antigemination hypotheses. The idea of describing morphological complexes as aligned clitics is also highlighted as a promising avenue for future research, potentially bridging to syntax.
Summary updated