Introduction
State-of-the-art Scene Text Recognition (STR) models often struggle with "unseen" words—sequences of characters not present in the training vocabulary. While these models perform well on common words, their accuracy drops significantly when encountering rare terms, technical jargon, or brand names. This "vocabulary-locked" behavior stems from the single-pass nature of current architectures, which lack a mechanism for self-correction. In contrast, humans recognize unfamiliar words through iterative refinement. This paper proposes a meta-framework that introduces an iterative feedback loop to improve generalization to unseen character sequences.
Methodology
The proposed framework treats text recognition as a multi-step refinement process. The architecture is divided into three primary components:
- Main Network ($T$): Comprising a feature extractor ($T_A$) and a decoder ($T_B$).
- Feedback Module ($F$): A Cross-Modal Conditional Variational Autoencoder (CVAE) that serves as the core of the refinement process. It maps discrete text predictions from a previous step back into a continuous parameter space.
- Feedback Conditioning Layer: This layer uses parameters from the CVAE to modulate visual feature maps (via FiLM-like modulation) before they are re-processed by the decoder in the subsequent iteration.
A key innovation is the "Distilling from Errors" strategy. The feedback module is trained not only on ground-truth data but also on realistic error distributions ($\mathcal{G}$) collected from other high-performing models. By learning from these common OCR mistakes (e.g., confusing 'h' with 'n'), the CVAE develops a linguistic prior that guides the model from an incorrect prediction toward the correct one.
Experimental Setup
The model was evaluated on standard Scene Text Recognition (STR) datasets (IIIT5K, SVT, IC13, IC15, SVTP, CUTE80) and Handwriting Text Recognition (HTR) datasets (IAM, RIMES). To rigorously test generalization, the authors introduced a Disjoint Setup (DS), where the test set contains only words entirely absent from the training set. This setup provides a more realistic measure of a model's ability to "read" rather than memorize.
Results
The iterative framework demonstrates consistent performance improvements, particularly in the challenging Disjoint Setup.
Key findings include:
- Significant Gains in DS: On the IAM handwriting dataset, the framework achieved a 10.5% absolute increase in Word Recognition Accuracy (WRA) over the baseline. Similar gains were observed on RIMES (10.8%) and IC15 (6.9%).
- Iterative Refinement: Accuracy typically peaks at the second iteration ($t=2$), showing the effectiveness of the feedback loop.
- Efficiency: The feedback module is lightweight, adding only approximately 2.19ms of computational overhead per iteration.
- Versatility: The module acts as a "plug-and-play" meta-framework, successfully boosting the performance of various existing SOTA architectures like ASTER and SCATTER.
Discussion and Conclusion
While the iterative approach significantly improves recognition of unseen words, it faces diminishing returns beyond a few iterations and introduces some computational overhead compared to single-pass models. Furthermore, the feedback quality is tied to the representativeness of the error distribution $\mathcal{G}$. Despite these limitations, the framework represents a significant step toward more robust, human-like text recognition systems that can generalize beyond their training vocabulary.
Summary updated