Explicit Relational Reasoning Network for Scene Text Detection

Yuchen Su, Zhineng Chen, Yongkun Du, Zhilong Ji, Kai Hu, Jinfeng Bai, Xieping Gao

Abstract: Connected component (CC) is a proper text shape representation that aligns with human reading intuition. However, CC-based text detection methods have recently faced a developmental bottleneck that their time-consuming post-processing is difficult to eliminate. To address this issue, we introduce an explicit relational reasoning network (ERRNet) to elegantly model the component relationships without post-processing. Concretely, we first represent each text instance as multiple ordered text components, and then treat these components as objects in sequential movement. In this way, scene text detection can be innovatively viewed as a tracking problem. From this perspective, we design an end-to-end tracking decoder to achieve a CC-based method dispensing with post-processing entirely. Additionally, we observe that there is an inconsistency between classification confidence and localization quality, so we propose a Polygon Monte-Carlo method to quickly and accurately evaluate the localization quality. Based on this, we introduce a position-supervised classification loss to guide the task-aligned learning of ERRNet. Experiments on challenging benchmarks demonstrate the effectiveness of our ERRNet. It consistently achieves state-of-the-art accuracy while holding highly competitive inference speed.

Introduction

Scene text detection is a critical task in computer vision, generally divided into segmentation-based and regression-based approaches. Segmentation-based methods offer flexibility for curved text but are sensitive to noise, while regression-based methods are robust but often struggle with extreme scales and complex shapes.

Connected Component (CC)-based methods provide a compelling middle ground by representing text as a series of simple geometric parts. While these methods align well with human intuition, they have historically been hindered by a "developmental bottleneck": the requirement for computationally expensive and heuristic-heavy post-processing to group and order individual components. Presented at AAAI 2025, the Explicit Relational Reasoning Network (ERRNet) overcomes this by reframing detection as an end-to-end tracking problem.

Methodology

ERRNet introduces three primary innovations to eliminate post-processing and improve task alignment.

Tracking-based Formulation

Instead of detecting components and then grouping them via manual rules, ERRNet views each text instance as a sequence of components moving through "hypothetical frames." In this framework, scene text detection is innovatively treated as a tracking task.

Comparison between traditional CC-based pipelines (a) and the proposed ERRNet tracking-based formulation (b).

As shown in Figure 1, traditional methods require separate grouping, ordering, and merging steps. In contrast, ERRNet uses an Explicit Relational Reasoning (ERR) Decoder. This Transformer-based decoder employs intra-frame attention to model spatial relationships between different text instances and inter-frame attention to model the sequential relationship of components within a single instance. By using bipartite graph matching during training, the model learns to supervise the entire sequence at once, dispensing with post-processing entirely.

Polygon Monte-Carlo (PMC) and Task Alignment

A common issue in detection is the inconsistency between classification confidence and localization quality. To address this, the authors proposed the Polygon Monte-Carlo (PMC) method. Calculating the exact intersection of arbitrary polygons is mathematically complex and slow for training. PMC turns this into a counting problem by sampling 10,000 points within the shapes on a GPU, allowing for fast and accurate estimation of the Polygon Intersection over Union (PIoU). This PIoU then guides a position-supervised classification loss, ensuring the model assigns high confidence only to accurate localizations.

Experiments and Results

ERRNet was evaluated on several challenging benchmarks, including Total-Text, CTW1500, and ArT.

Quantitative Performance

ERRNet demonstrates a superior balance of speed and accuracy:

  • State-of-the-Art Accuracy: On the CTW1500 dataset, ERRNet achieved an F-measure of 89.4%, outperforming recent SOTA models like TextBPN++.
  • High Efficiency: It achieves an inference speed of 31.5 FPS on an RTX 3090. Notably, it is 16x faster than the previous leading CC-based method, DRRG, which was limited by its post-processing requirements.

Qualitative Comparison

Qualitative comparison of ERRNet against other state-of-the-art methods on a complex text instance.

Qualitative results (Figure 6) highlight ERRNet's ability to handle overlapping text and extremely curved instances. While other methods like TextBPN++ or DRRG may produce fragmented or imprecise boundaries on complex shapes like the "Texas" sign, ERRNet provides a more coherent and accurate detection.

Conclusion

ERRNet revitalizes CC-based text detection by replacing heuristic post-processing with explicit relational reasoning. By treating detection as a tracking problem and introducing the PMC method for task alignment, it achieves a new state-of-the-art in both accuracy and speed. Future work may involve extending this framework to Scene Text Spotting by integrating character-level features into the tracking sequence.

Summary updated

Abstract

Connected component (CC) is a proper text shape representation that aligns with human reading intuition. However, CC-based text detection methods have recently faced a developmental bottleneck that their time-consuming post-processing is difficult to eliminate. To address this issue, we introduce an explicit relational reasoning network (ERRNet) to elegantly model the component relationships without post-processing. Concretely, we first represent each text instance as multiple ordered text components, and then treat these components as objects in sequential movement. In this way, scene text detection can be innovatively viewed as a tracking problem. From this perspective, we design an end-to-end tracking decoder to achieve a CC-based method dispensing with post-processing entirely. Additionally, we observe that there is an inconsistency between classification confidence and localization quality, so we propose a Polygon Monte-Carlo method to quickly and accurately evaluate the localization quality. Based on this, we introduce a position-supervised classification loss to guide the task-aligned learning of ERRNet. Experiments on challenging benchmarks demonstrate the effectiveness of our ERRNet. It consistently achieves state-of-the-art accuracy while holding highly competitive inference speed.
BibTeX
@inproceedings{su2025explicit,
  author = {Yuchen Su and Zhineng Chen and Yongkun Du and Zhilong Ji and Kai Hu and Jinfeng Bai and Xieping Gao},
  title = {{Explicit Relational Reasoning Network for Scene Text Detection}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  url = {https://openreview.net/forum?id=006TbIzeMJ},
}
Figure