CentripetalText: An Efficient Text Instance Representation for Scene Text Detection

Tao Sheng, Jie Chen, Zhouhui LianNeurIPS 2021

TL;DR

CentripetalText (CT) is a novel, efficient representation for scene text detection that models arbitrary shapes by mapping external text pixels to internal "kernels" via learned centripetal shifts. By replacing rigid coordinate regression with a "Relaxed L1 Loss" and enabling parallelized post-processing, CT achieves a state-of-the-art balance between accuracy and speed (e.g., 86.3% F-measure at 40 FPS on the Total-Text dataset).

What Problem Does This Paper Address?

Detecting text in the wild is notoriously difficult because text isn't always in neat horizontal boxes; it curves, rotates, and stretches into extreme aspect ratios.

Current approaches generally fall into two camps:

  1. Component-free methods: These try to regress a bounding box or quadrangle. They are fast but fail miserably on curved or irregular text.
  2. Component-based methods: These break text into smaller parts (like pixels or segments) and then try to "glue" them back together. While flexible, they often rely on slow, iterative post-processing or rigid regression models that don't handle prediction errors well.

The authors identify a specific bottleneck: most models use a "Dirac delta distribution" for regression, meaning they try to predict an exact numerical value for every pixel. In the messy reality of scene text, this is fragile and makes the model struggle with generalization.

Methods

The authors propose CentripetalText (CT), which represents a text instance as a combination of a text kernel (the central "skeleton" of the text) and centripetal shifts (vectors pointing from the outer pixels toward the kernel).

Illustration of CentripetalText representing text regions via centripetal shifts pointing to a text kernel.

Key technical innovations include:

  • Centripetal Shift Mapping: Instead of pixels "expanding" outward (which is usually an iterative, slow process), every pixel in a text region is assigned a shift vector that points to its corresponding kernel.
  • Relaxed L1 Loss: This is the "secret sauce." Instead of punishing the model for not hitting an exact coordinate, the loss function treats any shift that lands the pixel anywhere within the correct kernel region as a "correct" prediction. This tolerance makes the model much more robust to small errors.
  • One-Step Reconstruction: Because the shifts are predicted for all pixels simultaneously, the grouping process can be handled via parallel matrix operations rather than iterative searching, significantly boosting inference speed.
  • CentripetalText Proposal Network (CPN): The authors adapted their detector into a proposal module that can be plugged into end-to-end "text spotters" (which both detect and read text), replacing older, less accurate modules like the Segmentation Proposal Network (SPN).

Datasets

The model was evaluated on several standard benchmarks for scene text:

  • SynthText: Used for large-scale pre-training (800k synthetic images).
  • Total-Text: Focused on curved text at the word level.
  • CTW1500: Focused on curved text at the text-line level (using 14-point polygons).
  • MSRA-TD500: A dataset for multi-oriented long text lines.
  • HUST-TR400: Used as additional training data for the MSRA-TD500 experiments.

Key Insights and Ideas

  • Shift vs. Expansion: Most segmentation-based detectors start at a kernel and "grow" outward. CT flips this: pixels "look" inward to find their kernel. This allows for much faster, non-iterative grouping during inference.
  • Embracing Uncertainty: The "Relaxed L1 Loss" acknowledges that in deep learning, "close enough" is often better than "perfect but fragile." By allowing a range of valid predictions, the model converges more easily and generalizes better to unseen text shapes.
  • Efficiency through Parallelism: By formulating the reconstruction as a matrix operation, the authors moved the heavy lifting of post-processing onto the GPU, allowing the model to maintain high frame rates even on complex datasets.

Key Results

CentripetalText demonstrates a superior trade-off between precision and speed compared to previous heavyweights like DB (Differentiable Binarization) and PAN (Pixel Aggregation Network).

Performance comparison of CentripetalText (CT) against PAN and DB on the Total-Text dataset.
  • Total-Text (Curved): Achieved an 86.3% F-measure at 40.0 FPS. For comparison, a faster version (CT-320) reached 93.2 FPS while still maintaining an 81.2% F-measure.
  • MSRA-TD500 (Multi-oriented): Achieved an 86.1% F-measure at 34.8 FPS, outperforming PAN by 4.1% in accuracy.
  • End-to-End Recognition: When integrated into Mask TextSpotter v3, the CPN module improved the final text recognition F-measure by 1.1% on curved text.

Limitations and Future Work

While the paper highlights the efficiency of the shift-based representation, it does not deeply explore how the model handles extremely dense text where kernels might be only a few pixels apart. In such cases, a slight error in shift prediction could potentially map a pixel to the wrong adjacent kernel.

The authors suggest that CentripetalText can serve as a "valuable and common representation" for the broader community, hinting that future work may involve applying this centripetal logic to other instance segmentation tasks beyond just text.

Summary updated

BibTeX
@inproceedings{sheng2021centripetaltext,
  author = {Tao Sheng and Jie Chen and Zhouhui Lian},
  title = {{CentripetalText: An Efficient Text Instance Representation for Scene Text Detection}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2021},
}
Figure