Abstract
Traditional scene text spotting relies on a two-stage "detect-then-recognize" pipeline using Region of Interest (RoI) operations. These operations are sensitive to detection accuracy and struggle with irregular text shapes. We present MANGO, a one-stage framework that replaces explicit RoI cropping with a Position-aware Mask Attention (PMA) module. MANGO maps text instances to specific feature channels based on their spatial grid location, enabling simultaneous recognition of all text in an image. Experimental results demonstrate state-of-the-art performance on curved text datasets while requiring only coarse bounding box supervision.
Introduction
Existing end-to-end scene text spotters typically rely on a two-stage framework: they first detect text regions and then use Region of Interest (RoI) operations (like RoI-Align or Bezier-Align) to crop and feed those regions into a sequence recognizer.
However, this creates a significant bottleneck. The recognition step becomes highly sensitive to the accuracy of the detection step—a bounding box that is too tight might crop out character edges, leading to recognition errors. Furthermore, adapting RoI operations to handle irregular, curved text requires complex geometric transformations and expensive, labor-intensive polygonal annotations. MANGO addresses this by rethinking text spotting as a direct "attending and reading" problem, completely bypassing explicit RoI cropping and the strict dependency on perfect text boundaries.
Methodology
MANGO (Mask AttentioN Guided One-stage text spotter) discards RoI cropping entirely. Instead, it uses a standard ResNet-50 and FPN backbone followed by a novel Position-aware Mask Attention (PMA) module.
Position-aware Mask Attention (PMA)
The core of MANGO is the PMA module, which operates through the following steps:
- Grid Mapping: The image is conceptually divided into an $S \times S$ grid.
- Channel Allocation: The PMA module uses dynamic convolutions to generate spatial attention masks for both whole text instances (Instance-level Mask Attention, IMA) and individual characters (Character-level Mask Attention, CMA). Crucially, MANGO maps different text instances to different feature map channels based on the grid cells they occupy.
- Sequence Decoding: Once the text features are isolated in their respective channels, they are batched together and fed into a lightweight sequence decoder (a BiLSTM followed by a fully connected layer) that predicts the character sequences for all text instances simultaneously.
- Localization & Inference: To identify which grid cells contain text during inference, a parallel centerline segmentation branch provides rough localization. Finally, a character-weighted voting strategy merges overlapping predictions from adjacent grid cells.
Experimental Results
MANGO was evaluated across several challenging datasets, including ICDAR2013/2015 for regular text and Total-Text/SCUT-CTW1500 for irregular, curved text.
- Irregular Text Performance: On Total-Text, MANGO achieved state-of-the-art results with 83.6% (full lexicon) accuracy at 4.3 FPS, outperforming previous methods like ABCNet. On SCUT-CTW1500, it surpassed previous advances by 4.6% (full lexicon, 78.7%).
- Robustness to Weak Labels: A key advantage of MANGO is its ability to learn from coarse annotations. When trained on Total-Text using only weak rectangular bounding boxes instead of tight polygons, accuracy only dropped slightly from 83.6% to 80.6%.
- Generalization: On the CCPD license plate dataset, a simplified MANGO achieved 96.9% Average Precision without using any position annotations during fine-tuning, demonstrating high flexibility.
Conclusion
MANGO introduces a paradigm shift in scene text spotting by replacing spatial cropping with channel-based mapping. This approach preserves global spatial context, handles irregular text naturally, and significantly reduces the need for precise, labor-intensive annotations. While the model currently relies on a centerline segmentation branch for position hints in complex scenes, future work aims to achieve true sequence-only supervision in the wild.
Summary updated