Russian Stress Prediction using Maximum Entropy Ranking

Keith B. Hall, Richard SproatEMNLP 2013 · DOI

Abstract

Predicting lexical stress in Russian is a complex task due to its "free" and "mobile" nature, where stress can fall on any syllable and shift during inflection. This paper presents a high-performance system that frames stress prediction as a Maximum Entropy (MaxEnt) ranking problem. By combining local character-level features with linguistically motivated global features, such as affixes and abstract phonetic classes, the model achieves over 98% accuracy on known word roots and significantly improves generalization to entirely new vocabulary.

Introduction: The Complexity of Russian Stress

In many languages, stress follows regular rules. In Russian, however, stress is unpredictable for non-native speakers and automated systems alike. It is "free" (not tied to a specific syllable position) and "mobile" (shifting between singular and plural forms). Correct stress is vital because it fundamentally alters vowel pronunciation through reduction; for instance, an unstressed 'o' may sound like an 'a'.

Figure 1: Examples of mobile stress in Russian nouns across different accentual paradigms (acc, unacc, postacc).

For technologies like Text-to-Speech (TTS), incorrect stress results in unnatural or unintelligible speech. This research aims to bridge the gap between rule-based linguistic systems and data-driven machine learning to provide a robust solution for Russian stress prediction.

Methodology

The authors treat stress prediction as a ranking task rather than a simple classification problem. Given a word, the model generates all possible stress patterns—including those with secondary stress—and ranks them using a Maximum Entropy (MaxEnt) framework.

Feature Engineering

The model utilizes several layers of features to capture the nuances of the language:

  • Local Context: Character trigrams (vowel + surrounding consonants) capture local phonetic cues.
  • Global Affixes: Since Russian stress is heavily tied to morphology, the model includes every possible prefix and suffix as a feature.
  • Abstract Phonetic Classes: Characters are mapped to categories like "stop," "nasal," or "fricative" to help the model generalize based on the word's structural "shape."
  • Ambiguity Handling: The model explicitly accounts for the letter ё, which is always stressed but often written as е in standard text.

Datasets and Evaluation

The system was trained on a massive dataset of approximately 1.9 million fully inflected word forms derived from Zaliznyak’s Grammatical Dictionary of the Russian Language. Evaluation was conducted on two distinct sets:

  1. Standard Test Set: 100,000 word forms, which may share roots (lemmata) with the training data.
  2. "No Shared Lemmata" Set: A specialized set of 248 forms with roots entirely absent from the training data, serving as a rigorous test for generalization.

Results

The inclusion of global morphological features and abstract classes provided a significant performance boost over baseline models using only local features.

Figure 4: Accuracy results comparing different feature sets across shared and unseen lemmata.

As shown in Figure 4, the combined model reached 98.7% accuracy on the standard test set. More importantly, on the difficult "no shared lemmata" set, accuracy reached 83.9%, a substantial improvement over the 80.6% baseline. This demonstrates that the model successfully learned underlying linguistic logic rather than merely memorizing word roots. Furthermore, the use of MaxEnt and Stochastic Gradient Descent (SGD) ensures the system is efficient enough for real-time production environments.

Conclusion

The success of this MaxEnt ranking approach underscores the importance of morphology in Russian stress prediction. By integrating global affix features and phonetic abstractions, the model achieves high accuracy and robust generalization. Future work may involve expanding the evaluation on rare loanwords and refining the handling of specific linguistic categories like postaccented nouns.

Summary updated

BibTeX
@inproceedings{hall2013russian,
  author = {Keith B. Hall and Richard Sproat},
  title = {{Russian Stress Prediction using Maximum Entropy Ranking}},
  booktitle = {Conference on Empirical Methods in Natural Language Processing},
  year = {2013},
  doi = {10.18653/v1/d13-1088},
}
Figure