Introduction
Part-of-speech (POS) tagging is a foundational task in Natural Language Processing (NLP), but it presents significant challenges for morphologically rich languages like Russian. Unlike English, where a word might have only a few forms, Russian words change significantly based on case, gender, number, and person. This results in a complex "tagset" containing hundreds or thousands of possible combinations.
Building a high-quality tagger typically requires a large "Gold Standard" corpus—thousands of pages of text manually labeled by experts. For many languages, such resources are unavailable. Researchers at Ohio State University addressed this "resource gap" by investigating whether a well-resourced, linguistically related language like Czech could be used to train a tagger for an under-resourced one like Russian.
Methodology
The researchers developed a multi-pronged strategy to bridge the gap between Czech and Russian linguistic structures:
- Cross-Language Training: They utilized the TnT tagger, a statistical model based on second-order Markov Models. Instead of training on Russian data, they trained the model's transition probabilities (the likelihood of one tag following another) on a 1.5-million-token Czech corpus from the Prague Dependency Treebank.
- Resource-light Morphological Analysis: To handle vocabulary differences, they built an analyzer using 80 linguistic "paradigms" (inflectional patterns) to guess potential tags for Russian words based on their suffixes.
- Tagset Decomposition: Russian tags are represented as 15-symbol strings. To combat data sparsity, the team broke these down into individual attributes like Case, Gender, and Number. They trained separate "sub-taggers" for these attributes and used a voting mechanism to determine the final tag.
- Ambiguity Reduction: The morphological analyzer often generates over 10 possible tags per word. To refine these guesses, the authors implemented Longest Ending Filtering (LEF)—a heuristic that prioritizes the longest matching suffix—and automatically derived a Russian lexicon from the 1-million-token Uppsala Russian Corpus.
Key Results
The system was evaluated using a hand-annotated portion of the Russian translation of George Orwell’s 1984. The results demonstrated that linguistic proximity can indeed serve as a powerful proxy for training data.
- Baseline Performance: Randomly selecting from the morphological analyzer's guesses yielded only 33.6% accuracy.
- Impact of Heuristics: Implementing LEF and a large unannotated lexicon significantly improved performance. As shown in the data, the combination of these methods brought tagging accuracy to 69.4%.
- Optimized System: By applying "Russification" (modifying Czech training data to better match Russian syntax) and the sub-tagger voting system, final accuracy reached 73.5%. In a head-to-head comparison on a small test set, the optimized system achieved 88% accuracy, outperforming a contemporary commercial tagger from Xerox (82%).
Discussion and Insights
The core insight of this research is that while the vocabularies of Czech and Russian differ, the underlying structural patterns of how parts of speech interact are remarkably similar. This allows for a "warm start" in NLP tasks for under-resourced languages.
However, limitations remain. The 73.5% accuracy is lower than the ~95% typically seen in same-language training. The approach is also highly dependent on the genetic relationship between the source and target languages; it is unlikely to be as effective for unrelated language pairs. Future work may involve more sophisticated "cognate detection" to automatically align the vocabularies of the two languages, further reducing the need for manual annotation.
Summary updated