Abstract
Scene text spotting is essential in various computer vision applications, enabling extracting and interpreting textual information from images. However, existing methods often neglect the spatial semantics of word images, leading to suboptimal detection recall rates for long and short words within long-tailed word length distributions that exist prominently in dense scenes. In this paper, we present WordLenSpotter, a novel word length-aware spotter for scene text image detection and recognition, improving the spotting capabilities for long and short words, particularly in the tail data of dense text images. We first design an image encoder equipped with a dilated convolutional fusion module to integrate multiscale text image features effectively. Then, leveraging the Transformer framework, we synergistically optimize text detection and recognition accuracy after iteratively refining text region image features using the word length prior. Specially, we design a Spatial Length Predictor module (SLP) using character count prior tailored to different word lengths to constrain the regions of interest effectively. Furthermore, we introduce a specialized word Length-aware Segmentation (LenSeg) proposal head, enhancing the network's capacity to capture the distinctive features of long and short terms within categories characterized by long-tailed distributions. Comprehensive experiments on public datasets and our dense text spotting dataset DSTD1500 demonstrate the superiority of our proposed methods, particularly in dense text image detection and recognition tasks involving long-tailed word length distributions encompassing a range of long and short words.
Introduction
Scene text spotting—the combined task of detecting and recognizing text in images—has seen massive leaps thanks to Transformer-based architectures. However, most models still stumble when faced with "dense" scenes, such as a page of a textbook or a crowded menu. In these environments, text follows a long-tailed distribution: most words are of average length (4–10 characters), while very short or very long words are rare and frequently missed.
The Long-Tail Problem in Dense Text
Traditional text spotters often treat all text instances equally, but real-world data is imbalanced. In dense scenes, the "head" of the distribution (medium-length words) dominates the training signal, causing the model to neglect the "tail" (words with 1–3 or 10+ characters). Furthermore, dense text images require a large receptive field to distinguish between tightly packed lines of text. Existing methods often lack the spatial semantic awareness needed to accurately bound these instances, leading to low recall rates in complex, high-density environments.
WordLenSpotter Architecture
WordLenSpotter builds on the Transformer-based spotting paradigm but introduces three key architectural innovations to incorporate word length as a structural prior:
- Enhanced Image Encoder: The model uses a Swin Transformer paired with a BiFPN (Bidirectional Feature Pyramid Network) and a Dilated Convolutional Fusion module. This expands the receptive field, allowing the network to better integrate multiscale features—essential for spotting tiny text nestled within large blocks.
- Spatial Length Predictor (SLP): This module exploits the strong positive correlation between a word's aspect ratio and its character count. By predicting these spatial semantics early, the model can better constrain the Regions of Interest (RoI) for the decoder.
- Length-aware Segmentation (LenSeg) Head: To specifically target the "tail" data, the authors designed a proposal head that categorizes text into background, regular length, and extreme length. This forces the network to learn distinctive features for those tricky short and long terms.
Experiments and Results
The authors evaluated WordLenSpotter on several standard benchmarks, including ICDAR2015, Total-Text, and CTW1500. Most importantly, they introduced a new dataset: DSTD1500 (Dense Scene Text Dataset). It contains 1,500 images from daily reading scenarios with over 75,000 English text instances, averaging more than 50 instances per image.
WordLenSpotter achieved state-of-the-art (SOTA) results on the new dense dataset:
- DSTD1500: Achieved an F-score of 83.30% in detection and 81.95% in end-to-end spotting, outperforming previous SOTA models.
- Total-Text: Reached an F-score of 86.6% for end-to-end spotting.
- Ablation Studies: Removing the SLP or LenSeg modules led to a measurable drop in performance, confirming that the "word length-aware" components are critical for dense scenes.
Conclusion
WordLenSpotter demonstrates that treating word length as a spatial prior significantly improves spotting accuracy for extreme word lengths in dense environments. While the model is computationally intensive—running at approximately 2.2 FPS on an NVIDIA Tesla A800—it provides a robust solution for complex document interpretation. Future work will focus on optimizing inference speed and improving label generation for curved text shapes.
Summary updated