RSCA: Real-Time Segmentation-Based Context-Aware Scene Text Detection

Jiachen Li, Yuan Lin, Rongrong Liu, Chiu Man Ho, Humphrey Shi

Abstract

Segmentation-based scene text detection methods have been widely adopted for arbitrary-shaped text detection recently, because they make accurate pixel-level predictions on curved text instances and facilitate real-time inference without time-consuming processing on anchors. However, current segmentation-based models are unable to learn the shapes of curved texts and often require complex label assignments or repeated feature aggregations for more accurate detection. In this paper, we propose RSCA: a Real-time Segmentation-based Context-Aware model for arbitrary-shaped scene text detection, which sets a strong baseline for arbitrary-shaped scene text detection with two simple yet effective strategies: Local Context-Aware Upsampling and Dynamic Text-Spine Labeling, which model local spatial transformation and simplify label assignments separately. Based on these strategies, RSCA achieves state-of-the-art performances without complex label assignments or repeated feature aggregations in a real-time inference speed. We conduct extensive experiments on multiple benchmarks to validate the effectiveness of our method. RSCA-640 reaches 83.9% F-measure at 48.3 FPS on CTW1500 dataset.

Introduction

Detecting text in natural scenes is a fundamental computer vision task, yet it remains challenging due to the diverse shapes, orientations, and scales of text instances. While segmentation-based methods excel at handling arbitrary-shaped text by making pixel-level predictions, they often suffer from two main limitations. First, standard neural network operations like convolution and pooling use fixed geometric structures, struggling to model the irregular spatial transformations inherent in curved text. Second, achieving high accuracy often necessitates complex, hand-crafted label assignments or computationally expensive repeated feature aggregations.

Proposed Methodology: RSCA

RSCA addresses these challenges through a streamlined architecture that prioritizes both accuracy and speed. The model utilizes a standard ResNet-50 backbone and a Feature Pyramid Network (FPN) structure, enhanced by two key innovations:

Local Context-Aware Upsampling (LCAU)

Traditional upsampling methods, such as bilinear interpolation, rely on fixed mathematical formulas that do not account for image content. RSCA introduces LCAU, which replaces standard upsampling with a content-aware module. LCAU employs a small convolution to generate a local attention weight matrix, allowing the model to adaptively upsample pixels based on their surrounding context. This approach effectively captures the flow of curved text without the high computational overhead of global self-attention mechanisms.

The RSCA architecture featuring Local Context-Aware Upsampling (LCAU) modules.

Dynamic Text-Spine Labeling (DTSL)

Most segmentation models train using a "shrunk" version of the text, known as a text spine, to separate adjacent text instances. While typical models use a fixed shrink ratio, RSCA employs Dynamic Text-Spine Labeling (DTSL). This strategy begins training with a very thin spine—an "easy" target—and gradually expands it toward the full text boundary as training progresses. This curriculum learning approach helps the model build stable internal representations of text before tackling the noisier boundary pixels.

Experimental Results

RSCA was evaluated on several challenging benchmarks, including CTW1500, Total-Text, MSRA-TD500, and ICDAR-2015. The results demonstrate an exceptional balance between precision and real-time performance.

On the CTW1500 dataset, which focuses on curved text, RSCA-640 achieved an 83.9% F-measure at 48.3 FPS. A higher-resolution variant, RSCA-800, reached an 85.0% F-measure. The model also performed strongly on multi-oriented text, achieving an 88.4% F-measure on MSRA-TD500 while maintaining nearly 30 FPS.

Performance comparison of RSCA against other state-of-the-art methods on the CTW1500 dataset.

Conclusion

RSCA proves that high-performance scene text detection can be achieved through architectural simplicity rather than brute-force feature aggregation. By modeling local spatial transformations during upsampling and utilizing a dynamic labeling strategy, RSCA sets a new baseline for real-time, arbitrary-shaped text detection. Future work may explore further optimizations for mobile deployment and the development of more sophisticated loss functions tailored for scene text boundaries.

Summary updated

Abstract

Segmentation-based scene text detection methods have been widely adopted for arbitrary-shaped text detection recently, because they make accurate pixel-level predictions on curved text instances and facilitate real-time inference without time-consuming processing on anchors. However, current segmentation-based models are unable to learn the shapes of curved texts and often require complex label assignments or repeated feature aggregations for more accurate detection. In this paper, we propose RSCA: a Real-time Segmentation-based Context-Aware model for arbitrary-shaped scene text detection, which sets a strong baseline for arbitrary-shaped scene text detection with two simple yet effective strategies: Local Context-Aware Upsampling and Dynamic Text-Spine Labeling, which model local spatial transformation and simplify label assignments separately. Based on these strategies, RSCA achieves state-of-the-art performances without complex label assignments or repeated feature aggregations in a real-time inference speed. We conduct extensive experiments on multiple benchmarks to validate the effectiveness of our method. RSCA-640 reaches 83.9% F-measure at 48.3 FPS on CTW1500 dataset.
BibTeX
@inproceedings{li2021rsca,
  author = {Jiachen Li and Yuan Lin and Rongrong Liu and Chiu Man Ho and Humphrey Shi},
  title = {{RSCA: Real-Time Segmentation-Based Context-Aware Scene Text Detection}},
  booktitle = {Workshops @ CVPR},
  year = {2021},
  url = {https://openreview.net/forum?id=l0Z4AXg6pb},
}
Figure