MANGO: A Mask Attention Guided One-Stage Scene Text Spotter

Liang Qiao, Ying Chen, Zhanzhan Cheng, Yunlu Xu, Yi Niu, Shiliang Pu, Fei Wu

Abstract

Traditional scene text spotting relies on a two-stage "detect-then-recognize" pipeline using Region of Interest (RoI) operations. These operations are sensitive to detection accuracy and struggle with irregular text shapes. We present MANGO, a one-stage framework that replaces explicit RoI cropping with a Position-aware Mask Attention (PMA) module. MANGO maps text instances to specific feature channels based on their spatial grid location, enabling simultaneous recognition of all text in an image. Experimental results demonstrate state-of-the-art performance on curved text datasets while requiring only coarse bounding box supervision.

Introduction

Existing end-to-end scene text spotters typically rely on a two-stage framework: they first detect text regions and then use Region of Interest (RoI) operations (like RoI-Align or Bezier-Align) to crop and feed those regions into a sequence recognizer.

However, this creates a significant bottleneck. The recognition step becomes highly sensitive to the accuracy of the detection step—a bounding box that is too tight might crop out character edges, leading to recognition errors. Furthermore, adapting RoI operations to handle irregular, curved text requires complex geometric transformations and expensive, labor-intensive polygonal annotations. MANGO addresses this by rethinking text spotting as a direct "attending and reading" problem, completely bypassing explicit RoI cropping and the strict dependency on perfect text boundaries.

Comparison between traditional RoI-based methods (a) and MANGO's mask attention approach (b).

Methodology

MANGO (Mask AttentioN Guided One-stage text spotter) discards RoI cropping entirely. Instead, it uses a standard ResNet-50 and FPN backbone followed by a novel Position-aware Mask Attention (PMA) module.

The MANGO architecture featuring the Position-aware Mask Attention (PMA) module and sequence recognizer.

Position-aware Mask Attention (PMA)

The core of MANGO is the PMA module, which operates through the following steps:

  1. Grid Mapping: The image is conceptually divided into an $S \times S$ grid.
  2. Channel Allocation: The PMA module uses dynamic convolutions to generate spatial attention masks for both whole text instances (Instance-level Mask Attention, IMA) and individual characters (Character-level Mask Attention, CMA). Crucially, MANGO maps different text instances to different feature map channels based on the grid cells they occupy.
  3. Sequence Decoding: Once the text features are isolated in their respective channels, they are batched together and fed into a lightweight sequence decoder (a BiLSTM followed by a fully connected layer) that predicts the character sequences for all text instances simultaneously.
  4. Localization & Inference: To identify which grid cells contain text during inference, a parallel centerline segmentation branch provides rough localization. Finally, a character-weighted voting strategy merges overlapping predictions from adjacent grid cells.

Experimental Results

MANGO was evaluated across several challenging datasets, including ICDAR2013/2015 for regular text and Total-Text/SCUT-CTW1500 for irregular, curved text.

  • Irregular Text Performance: On Total-Text, MANGO achieved state-of-the-art results with 83.6% (full lexicon) accuracy at 4.3 FPS, outperforming previous methods like ABCNet. On SCUT-CTW1500, it surpassed previous advances by 4.6% (full lexicon, 78.7%).
  • Robustness to Weak Labels: A key advantage of MANGO is its ability to learn from coarse annotations. When trained on Total-Text using only weak rectangular bounding boxes instead of tight polygons, accuracy only dropped slightly from 83.6% to 80.6%.
  • Generalization: On the CCPD license plate dataset, a simplified MANGO achieved 96.9% Average Precision without using any position annotations during fine-tuning, demonstrating high flexibility.

Conclusion

MANGO introduces a paradigm shift in scene text spotting by replacing spatial cropping with channel-based mapping. This approach preserves global spatial context, handles irregular text naturally, and significantly reduces the need for precise, labor-intensive annotations. While the model currently relies on a centerline segmentation branch for position hints in complex scenes, future work aims to achieve true sequence-only supervision in the wild.

Summary updated

Abstract

Recently end-to-end scene text spotting has become a popular research topic due to its advantages of global optimization and high maintainability in real applications. Most methods attempt to develop various region of interest (RoI) operations to concatenate the detection part and the sequence recognition part into a two-stage text spotting framework. However, in such framework, the recognition part is highly sensitive to the detected results (e.g., the compactness of text contours). To address this problem, in this paper, we propose a novel Mask AttentioN Guided One-stage text spotting framework named MANGO, in which character sequences can be directly recognized without RoI operation. Concretely, a position-aware mask attention module is developed to generate attention weights on each text instance and its characters. It allows different text instances in an image to be allocated on different feature map channels which are further grouped as a batch of instance features. Finally, a lightweight sequence decoder is applied to generate the character sequences. It is worth noting that MANGO inherently adapts to arbitrary-shaped text spotting and can be trained end-to-end with only coarse position information (e.g., rectangular bounding box) and text annotations. Experimental results show that the proposed method achieves competitive and even new state-of-the-art performance on both regular and irregular text spotting benchmarks, i.e., ICDAR 2013, ICDAR 2015, Total-Text, and SCUT-CTW1500.
BibTeX
@inproceedings{qiao2021mango,
  author = {Liang Qiao and Ying Chen and Zhanzhan Cheng and Yunlu Xu and Yi Niu and Shiliang Pu and Fei Wu},
  title = {{MANGO: A Mask Attention Guided One-Stage Scene Text Spotter}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2021},
  doi = {10.1609/aaai.v35i3.16348},
}
Figure