TL;DR
This paper tackles the challenge of OCR for Manchu, a low-resource language where complex word structures make standard dictionaries insufficient. The authors demonstrate that while expanding a lexicon with linguistically generated word forms improves coverage, it eventually hurts accuracy by introducing too many "competitor" words; instead, a "confidence fallback" strategy—relying on the dictionary when certain and falling back to raw model predictions when not—yields the best results.
What Problem Does This Paper Address?
Optical Character Recognition (OCR) for low-resource languages like Manchu is notoriously difficult. Manchu uses a vertical, cursive script, and there is very little annotated data available for training deep learning models.
A major hurdle is Manchu’s agglutinative nature: words are built by attaching multiple suffixes to a root. This creates a massive variety of possible word forms, many of which will never appear in a training set. While character-level OCR models are good at "near misses," they often make small spelling errors. We can fix these by forcing the model to choose from a lexicon (a dictionary of valid words), but if the correct word isn't in that lexicon (a "coverage" problem), the system is guaranteed to fail.
The authors ask: Should we just make the lexicon as big as possible by generating all possible root-suffix combinations? Or is there a point where a giant dictionary does more harm than good?
Methods
The researchers propose a multi-pronged approach to optimize lexicon-constrained decoding:
- Morphological Expansion: They use a base dictionary to segment training text into roots and suffix chains. They then generate ~510,000 new candidate words by pairing every seen root with every seen suffix chain.
- Frequency Ranking: Not all generated words are equally likely. They rank candidates based on the frequency of their components in the training data and test different expansion sizes ($N$).
- Coverage–Precision Decomposition: They introduce a formal way to analyze accuracy as the product of coverage (is the word in the lexicon?) and in-lexicon precision (can the model pick the right word from the candidates?).
- Confidence Fallback: Instead of a "hard" lexicon constraint, they implement a "soft" one. The system performs both lexicon-constrained decoding and "greedy" (open) decoding. If the model is much more confident in its greedy prediction than the best dictionary match, it "falls back" to the greedy result.
Datasets
The study uses a mix of printed and handwritten Manchu text:
- Manchu50000: A collection of printed text with regular glyph forms.
- Handwritten Archives: Three collections (MengGuTangDang, SuiJunJiXing, and Junjichu) featuring significant variation in handwriting and ink quality.
- Test Tiers: To measure generalization, they split the test set into tiers: Tier A (words seen in training), Tier B (seen roots in new combinations), and Tier C (entirely unseen roots).
Key Insights and Ideas
- The Precision Tax: Expanding the lexicon is not a free lunch. As you add more words to the dictionary to improve coverage, you increase the number of "plausible competitors." This makes it harder for the OCR model to distinguish the correct word from a similar-looking but incorrect entry in the expanded list.
- Frequency is a Vital Signal: When expanding the lexicon, ranking candidates by the frequency of their roots and suffixes is essential. Randomly adding valid morphological combinations performs significantly worse because it adds too much "noise" to the candidate space.
- Fallback Beats Brute Force: The "aha moment" of the paper is that a smaller, high-quality dictionary combined with a confidence-based fallback mechanism outperforms a massive, all-encompassing dictionary. Fallback acts as a safety valve, allowing the model to handle words it hasn't seen without cluttering the search space for words it does know.
Key Results
The authors tested their methods using both CRNN and Transformer-based recognizers.
- Lexicon Expansion: Adding 250,000 generated words to a base dictionary of ~29,000 words improved type accuracy on unseen combinations (Tier B) from 63.6% to 86.3%.
- The Winning Strategy: The best overall performance came from the base dictionary with confidence fallback. For the CRNN model, this reached 76.1% type accuracy and 85.5% token accuracy, outperforming even the best expanded lexicons.
- Oracle Insights: Even with "Oracle" routing (knowing perfectly when to use the dictionary vs. greedy decoding), the base dictionary still yielded higher accuracy than the expanded one, confirming that the "precision cost" of a giant lexicon is a fundamental limitation.
Limitations and Future Work
The study focuses on a single language (Manchu) and uses a relatively simple trigram character language model. The authors note that a stronger language model might narrow the gap between open and constrained decoding.
Furthermore, the test images share publication sources with the training data; future work is needed to see how well these strategies generalize to entirely new collections or different writers. The authors plan to extend these findings to other agglutinative scripts, such as traditional Mongolian, to see if the "coverage vs. precision" trade-off holds across different linguistic contexts.
Summary updated