TL;DR
This paper introduces the first stochastic finite-state morphological parser for Turkish, enabling probabilistic analysis of word forms. This advancement is crucial for tasks like spelling correction and language modeling in morphologically rich languages, where traditional methods struggle with high out-of-vocabulary rates.
What Problem Does This Paper Address?
Turkish is a highly agglutinative language, meaning words can be formed by adding many suffixes to a root, leading to a vast number of possible word forms. Traditional morphological parsers, while effective at analyzing word structure, typically output deterministic analyses. This is insufficient for applications requiring probability estimates over word forms, such as ranking spelling suggestions or building language models for speech recognition. Estimating probabilities directly on a static vocabulary is impractical due to the sheer number of possible word forms and high out-of-vocabulary rates. The paper addresses this by developing a parser that can assign probabilities to word analyses, bridging the gap between rule-based morphology and statistical language processing.
Methods
The core of the work is building a stochastic finite-state morphological parser. This is achieved by:
- Two-Level Morphology Formalism: The parser is based on the established two-level morphology formalism, using finite-state transducers (FSTs) to model phonological rules and morphotactics (word formation rules).
- Composition of Transducers: The morphological parser is formed by composing a morphophonemics transducer (handling sound changes) with a morphotactics transducer (handling word structure).
- Stochastization: The key innovation is making the morphotactics transducer stochastic. This involves:
- Disambiguation: Using a large, disambiguated text corpus (200 million words) to determine the most likely morphosyntactic analyses for words.
- Parameter Estimation: Employing a specialized semiring to count arc traversals in the disambiguated paths. The weights (probabilities) of arcs in the morphotactics transducer are then set proportional to these counts, effectively learning a statistical model of word formation. Add-one smoothing is used to avoid zero probabilities.
- OpenFST Library: The OpenFST weighted finite-state transducer library is used for all FST operations.
Datasets
- Lexicon: A new root lexicon of 55,278 words was compiled from the Turkish Language Institution dictionary.
- Training Corpus: A text corpus of 200 million words was collected from online newspapers. This corpus was used for stochastizing the morphotactics transducer.
- Test Set (Spelling Correction): A manually chosen test set containing 225 correct words with complex morphology and 43 commonly misspelled words.
- Test Set (Language Modeling): 3.1 hours of broadcast news speech data (2,410 utterances).
Key Insights and Ideas
- Probabilistic Morphology for Rich Languages: The paper demonstrates that it's feasible and beneficial to build probabilistic morphological parsers for highly agglutinative languages like Turkish, overcoming the limitations of static vocabularies.
- Leveraging Disambiguation for Training: Instead of relying on complex EM algorithms for parameter estimation, the authors cleverly use a pre-trained morphological disambiguator to select single, likely paths through the morphotactics transducer. This simplifies the training process significantly.
- Unified Framework for Applications: The stochastic parser provides a single, probabilistic output that can be directly used for diverse applications like ranking spelling suggestions and improving language models.
- Efficiency: The parser can analyze approximately 8700 words per second on a standard processor, making it practical for real-world use.
Key Results
- Spelling Correction: The stochastic parser is used to rank spelling suggestions. The system achieves Precision and Recall scores of 0.81 and 0.93, respectively, on a test set.
- Language Modeling: A morphology-based unigram language model derived from the parser shows a significantly lower Out-Of-Vocabulary (OOV) rate (2.8%) compared to standard word-based models (7% for 50K vocabulary, 4.7% for 100K vocabulary). While the Word Error Rate (WER) was initially higher for the pure morphology model, a combined "word+morphology" model performed competitively.
- Performance: The parser can analyze about 96.7% of tokens in the 200 million-word corpus.
Limitations and Future Work
- Ambiguity in Recognition: The paper notes that the morphological parser can still be ambiguous, which can hinder direct optimization for speech recognition tasks. The "word+morphology" model was proposed to mitigate this.
- Contextual Information: For spelling correction, the current approach primarily uses unigrams. The authors suggest incorporating morpheme-based language models to leverage contextual information.
- Edit Distance: The spelling correction currently limits edit distance to 1, but the authors state it's straightforward to extend this.
- Future Work: The authors plan to train a language model over morphological features and use it to rescore hypotheses generated by the morphology-based models on-the-fly in speech recognition.
Summary updated