Coverage Governs Accuracy: When to Expand the Lexicon and When to Fall Back in Low-Resource Manchu OCR

Ruixin Yu, Zifan Wang, Mingxin Ding, Xiaoxin Sun, Dongjie Wang, Suhua Wang, Minghao YinarXiv 2026 · arXiv

TL;DR

This paper tackles the challenge of OCR for Manchu, a low-resource language where complex word structures make standard dictionaries insufficient. The authors demonstrate that while expanding a lexicon with linguistically generated word forms improves coverage, it eventually hurts accuracy by introducing too many "competitor" words; instead, a "confidence fallback" strategy—relying on the dictionary when certain and falling back to raw model predictions when not—yields the best results.

What Problem Does This Paper Address?

Optical Character Recognition (OCR) for low-resource languages like Manchu is notoriously difficult. Manchu uses a vertical, cursive script, and there is very little annotated data available for training deep learning models.

A major hurdle is Manchu’s agglutinative nature: words are built by attaching multiple suffixes to a root. This creates a massive variety of possible word forms, many of which will never appear in a training set. While character-level OCR models are good at "near misses," they often make small spelling errors. We can fix these by forcing the model to choose from a lexicon (a dictionary of valid words), but if the correct word isn't in that lexicon (a "coverage" problem), the system is guaranteed to fail.

The authors ask: Should we just make the lexicon as big as possible by generating all possible root-suffix combinations? Or is there a point where a giant dictionary does more harm than good?

Methods

The researchers propose a multi-pronged approach to optimize lexicon-constrained decoding:

  1. Morphological Expansion: They use a base dictionary to segment training text into roots and suffix chains. They then generate ~510,000 new candidate words by pairing every seen root with every seen suffix chain.
  2. Frequency Ranking: Not all generated words are equally likely. They rank candidates based on the frequency of their components in the training data and test different expansion sizes ($N$).
  3. Coverage–Precision Decomposition: They introduce a formal way to analyze accuracy as the product of coverage (is the word in the lexicon?) and in-lexicon precision (can the model pick the right word from the candidates?).
  4. Confidence Fallback: Instead of a "hard" lexicon constraint, they implement a "soft" one. The system performs both lexicon-constrained decoding and "greedy" (open) decoding. If the model is much more confident in its greedy prediction than the best dictionary match, it "falls back" to the greedy result.
The coverage-precision plane shows that while expansion increases coverage, it often comes at the cost of precision. Fallback strategies offer a better trade-off.

Datasets

The study uses a mix of printed and handwritten Manchu text:

  • Manchu50000: A collection of printed text with regular glyph forms.
  • Handwritten Archives: Three collections (MengGuTangDang, SuiJunJiXing, and Junjichu) featuring significant variation in handwriting and ink quality.
  • Test Tiers: To measure generalization, they split the test set into tiers: Tier A (words seen in training), Tier B (seen roots in new combinations), and Tier C (entirely unseen roots).

Key Insights and Ideas

  • The Precision Tax: Expanding the lexicon is not a free lunch. As you add more words to the dictionary to improve coverage, you increase the number of "plausible competitors." This makes it harder for the OCR model to distinguish the correct word from a similar-looking but incorrect entry in the expanded list.
  • Frequency is a Vital Signal: When expanding the lexicon, ranking candidates by the frequency of their roots and suffixes is essential. Randomly adding valid morphological combinations performs significantly worse because it adds too much "noise" to the candidate space.
  • Fallback Beats Brute Force: The "aha moment" of the paper is that a smaller, high-quality dictionary combined with a confidence-based fallback mechanism outperforms a massive, all-encompassing dictionary. Fallback acts as a safety valve, allowing the model to handle words it hasn't seen without cluttering the search space for words it does know.
Figure 2: Effect of the number N of generated candidates. (a) Validation type and token accuracy as the top-N candidates are added to the dictionary; N ∗= 250k maximizes type accuracy. (b) Test token accuracy per tier for the dictionary, N ∗and the full union (three CRNN seeds, mean ± SD).

Key Results

The authors tested their methods using both CRNN and Transformer-based recognizers.

  • Lexicon Expansion: Adding 250,000 generated words to a base dictionary of ~29,000 words improved type accuracy on unseen combinations (Tier B) from 63.6% to 86.3%.
  • The Winning Strategy: The best overall performance came from the base dictionary with confidence fallback. For the CRNN model, this reached 76.1% type accuracy and 85.5% token accuracy, outperforming even the best expanded lexicons.
  • Oracle Insights: Even with "Oracle" routing (knowing perfectly when to use the dictionary vs. greedy decoding), the base dictionary still yielded higher accuracy than the expanded one, confirming that the "precision cost" of a giant lexicon is a fundamental limitation.
Detailed accuracy results across different tiers and decoding strategies.

Limitations and Future Work

The study focuses on a single language (Manchu) and uses a relatively simple trigram character language model. The authors note that a stronger language model might narrow the gap between open and constrained decoding.

Furthermore, the test images share publication sources with the training data; future work is needed to see how well these strategies generalize to entirely new collections or different writers. The authors plan to extend these findings to other agglutinative scripts, such as traditional Mongolian, to see if the "coverage vs. precision" trade-off holds across different linguistic contexts.

Summary updated

Abstract

Character-level OCR for low-resource Manchu often produces near-miss spelling errors. Lexicon-constrained decoding can correct such errors with lexical knowledge, but a fixed dictionary cannot cover the many unseen word forms produced by agglutinative word formation. We study whether morphological expansion alleviates this limitation, and how the expansion size and confidence fallback affect recognition. We segment the training transcriptions with a dictionary-based stem list, extract roots and suffix chains from them, generate about 510k candidates, select the expansion size through frequency ranking on the validation set, and decompose accuracy under hard lexicon constraints into the product of coverage and in-lexicon precision. Across three CRNN training seeds, adding 250k candidates to a base dictionary of about 29k words raises the type accuracy on unseen combinations of seen roots from 63.6% to 86.3%, and overall token accuracy from 82.4% to 84.2%. Adding all candidates lowers overall token accuracy to 81.5%, showing that the coverage gain comes with a loss of in-lexicon precision. A further comparison shows that the base dictionary with confidence fallback performs best overall among the CRNN systems tested, reaching 76.1% type and 85.5% token accuracy and outperforming the expanded lexicon both with and without fallback. A Transformer recognizer shows similar trends. These results indicate that morphological expansion relieves the constraint of a fixed dictionary, but a larger lexicon does not guarantee better recognition; in the present low-resource Manchu OCR setting, a selective combination of reliable lexical constraints and open recognition is more effective.
BibTeX
@misc{yu2026coverage,
  author = {Ruixin Yu and Zifan Wang and Mingxin Ding and Xiaoxin Sun and Dongjie Wang and Suhua Wang and Minghao Yin},
  title = {{Coverage Governs Accuracy: When to Expand the Lexicon and When to Fall Back in Low-Resource Manchu OCR}},
  year = {2026},
  eprint = {2611.23900},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.23900},
}

Loading related content...

Figure