Abstract
Segmentation-based scene text detection methods have been widely adopted for arbitrary-shaped text detection recently, since they make accurate pixel-level predictions on curved text instances and can facilitate real-time inference without time-consuming processing on anchors. However, current segmentation-based models are unable to learn the shapes of curved texts and often require complex label assignments or repeated feature aggregations for more accurate detection. In this paper, we propose RSCA: a Real-time Segmentation-based Context-Aware model for arbitrary-shaped scene text detection, which sets a strong baseline for scene text detection with two simple yet effective strategies: Local Context-Aware Upsampling and Dynamic Text-Spine Labeling, which model local spatial transformation and simplify label assignments separately. Based on these strategies, RSCA achieves state-of-the-art performance in both speed and accuracy, without complex label assignments or repeated feature aggregations. We conduct extensive experiments on multiple benchmarks to validate the effectiveness of our method. RSCA-640 reaches 83.9% F-measure at 48.3 FPS on CTW1500 dataset.
Introduction
Detecting text in natural scenes is a fundamental computer vision task, complicated by irregular, curved, or multi-oriented shapes. Modern approaches have largely pivoted from anchor-based bounding box detectors to segmentation-based methods. These methods classify text at the pixel level, allowing for flexible contours that better fit arbitrary shapes.
However, existing segmentation-based models encounter two primary bottlenecks. First, standard upsampling layers (e.g., bilinear or nearest-neighbor) lack spatial transformation modeling, making it difficult for networks to learn irregular geometries without heavy, repeated feature aggregation modules that hinder real-time performance. Second, training typically requires shrinking ground-truth polygons into "core" regions to avoid boundary ambiguity, a process that often necessitates tedious, dataset-specific tuning of "shrink ratios."
Methodology
The RSCA model addresses these challenges by integrating two novel strategies into a standard ResNet-based Feature Pyramid Network (FPN).
Local Context-Aware Upsampling (LCAU)
Instead of static upsampling, LCAU dynamically generates a local weight matrix based on the input feature map. By applying a small convolution followed by a softmax operation, the module computes context-aware weights. These weights are then multiplied with the original features, enabling the network to model local pixel-to-pixel relationships and spatial transformations. This approach provides the benefits of spatial modeling without the computational burden of global self-attention.
Dynamic Text-Spine Labeling (DTSL)
To simplify label assignment, RSCA introduces DTSL. Rather than using a fixed shrink ratio, DTSL adjusts the label size dynamically during training. It begins with a heavily shrunk "text spine"—representing the most unambiguous core of the text—and gradually expands the label toward the actual boundaries using a polynomial decay policy. This acts as a curriculum learning strategy, allowing the model to master easy regions before tackling difficult boundary pixels.
Experiments and Results
RSCA was evaluated on several benchmarks, including SynthText for pre-training, and CTW1500, Total-Text, MSRA-TD500, and ICDAR 2015 for testing.
The model demonstrates a superior speed-to-accuracy trade-off compared to previous state-of-the-art methods. On the CTW1500 curved text dataset, RSCA-640 achieved an 83.9% F-measure at 48.3 FPS, outperforming DBNet (83.4% at 22 FPS) and PAN (83.7% at 39.8 FPS). On MSRA-TD500, it reached an 86.0% F-measure at 52.5 FPS.
Ablation studies confirmed that LCAU outperforms standard upsampling and global attention mechanisms. Furthermore, DTSL improved the F-measure by approximately 1.6% over the best fixed-ratio labeling approach, proving that simplicity in design can outperform brute-force feature aggregation.
Discussion and Conclusion
RSCA proves that modeling spatial transformations within a local range is sufficient and efficient for reconstructing curved text. By removing the need for repeated feature pyramid aggregations, the model maintains high frame rates suitable for real-time applications.
While RSCA-640 is highly efficient, future work could explore more advanced loss functions to improve hard negative mining. Additionally, further optimization through model pruning and quantization could enhance performance on mobile CPUs, building on the promising results seen with the MobileNetV3 backbone variant.
Summary updated