MANGO: A Mask Attention Guided One-Stage Scene Text Spotter

Liang Qiao, Ying Chen, Zhanzhan Cheng, Yunlu Xu, Yi Niu, Shiliang Pu, Fei Wu

Overview

MANGO (Mask Attention Guided One-stage) is a novel scene text spotter that eliminates the need for traditional Region of Interest (RoI) operations like cropping or warping. By utilizing a Position-aware Mask Attention (PMA) module, it maps text instances directly to feature map channels, allowing for the end-to-end recognition of arbitrary-shaped text in a single, efficient pass.

The Limitations of Traditional RoI-Based Spotting

In scene text spotting—the combined task of detecting and recognizing text—the standard approach is a two-stage pipeline. First, a detector identifies a bounding box; second, an RoI operation (such as RoIAlign) crops and warps that region into a fixed shape for recognition.

Comparison between (a) traditional RoI-based text spotting and (b) MANGO's mask attention approach.

This conventional method suffers from three primary flaws:

  1. Error Accumulation: Imprecise detection boxes lead to recognition failures due to missing context or truncated characters.
  2. Architectural Complexity: Managing separate detection and recognition modules is computationally heavy and hinders global optimization.
  3. Irregular Text Handling: Curved or perspective-distorted text often requires complex "rectification" modules (like Thin-Plate Splines) to "un-distort" text before it can be read.

MANGO addresses these issues by treating text spotting as a direct "attending and reading" problem, bypassing the "crop-and-warp" bottleneck entirely.

Methodology: Position-aware Mask Attention (PMA)

The core of MANGO is the Position-aware Mask Attention (PMA) module, which enables the model to focus on text instances within the global feature map without explicit cropping.

The MANGO architecture featuring the Position-aware Mask Attention (PMA) module.

The PMA module consists of two primary components:

  • Instance-level Mask Attention (IMA): The image is divided into an $S \times S$ grid. Each grid cell corresponds to a specific channel in the feature map. Using dynamic convolutions, the model generates an attention mask for a text instance and assigns its features to the channel linked to its grid location.
  • Character-level Mask Attention (CMA): To refine recognition, this module generates fine-grained masks for individual characters. This provides precise spatial information for every letter, facilitating the handling of complex reading orders, such as vertical or curved text.

The architecture is completed by a Sequence Decoding Module (comprising a lightweight BiLSTM and a Fully Connected layer) and a parallel Centerline Segmentation branch. The segmentation branch identifies which grid cells contain valid text, guiding the inference process.

Key Insights

  • RoI-Free Robustness: By avoiding explicit cropping, MANGO retains global spatial context, making it robust to coarse localization. Perfect bounding boxes are no longer a prerequisite for perfect transcriptions.
  • Channel-as-Instance: Mapping spatial instances to specific feature channels (inspired by grid-based segmentation like SOLO) allows for the efficient, parallel recognition of multiple words in a single image.
  • Implicit Rectification: Instead of mathematically "flattening" text, MANGO learns to attend to characters in their natural sequence through the CMA module, effectively "reading" the curve rather than trying to straighten it.

Experimental Results

MANGO demonstrates state-of-the-art performance, particularly on irregular text benchmarks:

  • Total-Text: Achieved an F-measure of 72.9% (lexicon-free), significantly outperforming previous models like ABCNet (64.2%) and Mask TextSpotter (65.3%).
  • Efficiency: MANGO operates at approximately 8.9 FPS on Total-Text, nearly 4.5x faster than Mask TextSpotter (~2.0 FPS).
  • Generalization: The model proved effective on diverse datasets, including ICDAR 2013/2015 and the CCPD license plate dataset, even when trained with coarse rectangular bounding boxes rather than expensive polygonal masks.

Limitations

Despite its strengths, MANGO faces challenges with grid collisions, where multiple distinct text instances falling into a single grid cell may lead to missed detections. Additionally, the model's peak performance relies on pre-training with character-level annotations, which may be scarce for certain niche languages or fonts.

Summary updated

BibTeX
@inproceedings{qiao2021mango,
  author = {Liang Qiao and Ying Chen and Zhanzhan Cheng and Yunlu Xu and Yi Niu and Shiliang Pu and Fei Wu},
  title = {{MANGO: A Mask Attention Guided One-Stage Scene Text Spotter}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2021},
  doi = {10.1609/aaai.v35i3.16348},
}
Figure