Explicit Relational Reasoning Network for Scene Text Detection

Yuchen Su, Zhineng Chen, Yongkun Du, Zhilong Ji, Kai Hu, Jinfeng Bai, Xieping Gao

Abstract

Connected component (CC) is a proper text shape representation that aligns with human reading intuition. However, CC-based text detection methods have recently faced a developmental bottleneck that their time-consuming post-processing is difficult to eliminate. To address this issue, we introduce an explicit relational reasoning network (ERRNet) to elegantly model the component relationships without post-processing. Concretely, we first represent each text instance as multiple ordered text components, and then treat these components as objects in sequential movement. In this way, scene text detection can be innovatively viewed as a tracking problem. From this perspective, we design an end-to-end tracking decoder to achieve a CC-based method dispensing with post-processing entirely. Additionally, we observe that there is an inconsistency between classification confidence and localization quality, so we propose a Polygon Monte-Carlo method to quickly and accurately evaluate the localization quality. Based on this, we introduce a position-supervised classification loss to guide the task-aligned learning of ERRNet. Experiments on challenging benchmarks demonstrate the effectiveness of our ERRNet. It consistently achieves state-of-the-art accuracy while holding highly competitive inference speed.

Introduction

Scene text detection—the ability for AI to find and outline text in "wild" images like street signs or product labels—is a cornerstone of computer vision. However, it remains a tug-of-war between two approaches: segmentation (precise but messy and slow) and regression (fast but often inaccurate for complex shapes). Traditionally, researchers use Connected Component (CC) methods to detect text. These methods break a word down into smaller, manageable parts. While CC methods align well with human reading intuition, they suffer from a significant bottleneck: complex and time-consuming post-processing steps like grouping, ordering, and merging.

Methodology: Text Detection as Tracking

The core innovation of ERRNet is reframing text detection as a "tracking" problem. Instead of predicting all components simultaneously and stitching them together, ERRNet treats these components as objects in sequential movement across "time" (sequence order).

Comparison between traditional CC-based methods (a) and the proposed ERRNet tracking-based approach (b).

Explicit Relational Reasoning Decoder

The model utilizes a modulated Transformer decoder to model two types of relationships:

  1. Intra-frame Relationships: Spatial relationships between different text instances at the same sequence position.
  2. Inter-frame Relationships: Temporal relationships that maintain the internal order of components within a single text instance.

By using bipartite matching during training to supervise the entire sequence at once, the model learns to output clean, ordered text instances directly, eliminating the need for post-processing entirely.

Polygon Monte-Carlo (PMC) and Task Alignment

To address the inconsistency between classification confidence and localization quality, the authors developed a fast GPU-based Polygon Monte-Carlo method. This method calculates the Intersection over Union (IoU) of arbitrary polygons to weight the classification loss, ensuring the model is only confident when localization is accurate.

Experiments and Results

ERRNet was evaluated on several challenging benchmarks, including Total-Text, CTW1500, ArT, and MSRA-TD500.

Performance and Efficiency

ERRNet sets a new bar for performance and efficiency:

  • Accuracy: It achieved state-of-the-art results on CTW1500 with an F-measure of 89.4%, outperforming previous methods like DPText-DETR.
  • Speed: It achieves real-time inference at 31.5 FPS on an NVIDIA RTX 3090, a 16x speedup over previous leading CC-based methods like DRRG (2.0 FPS).
Qualitative result of ERRNet on curved scene text.

Conclusion

ERRNet demonstrates that treating the order of text components as a temporal dimension allows for efficient, end-to-end text detection without the overhead of traditional post-processing. While the current model uses a fixed sequence length, future work could explore variable lengths and extending this relational reasoning to full scene text understanding and visual question answering.

Reference: Su, Y., Chen, Z., Du, Y., et al. (2025). Explicit Relational Reasoning Network for Scene Text Detection. Proceedings of the Thirty-Ninth AAAI Conference on Artificial Intelligence (AAAI-25).

Summary updated

Abstract

Connected component (CC) is a proper text shape representation that aligns with human reading intuition. However, CC-based text detection methods have recently faced a developmental bottleneck that their time-consuming post-processing is difficult to eliminate. To address this issue, we introduce an explicit relational reasoning network (ERRNet) to elegantly model the component relationships without post-processing. Concretely, we first represent each text instance as multiple ordered text components, and then treat these components as objects in sequential movement. In this way, scene text detection can be innovatively viewed as a tracking problem. From this perspective, we design an end-to-end tracking decoder to achieve a CC-based method dispensing with post-processing entirely. Additionally, we observe that there is an inconsistency between classification confidence and localization quality, so we propose a Polygon Monte-Carlo method to quickly and accurately evaluate the localization quality. Based on this, we introduce a position-supervised classification loss to guide the task-aligned learning of ERRNet. Experiments on challenging benchmarks demonstrate the effectiveness of our ERRNet. It consistently achieves state-of-the-art accuracy while holding highly competitive inference speed.
BibTeX
@inproceedings{su2025explicit,
  author = {Yuchen Su and Zhineng Chen and Yongkun Du and Zhilong Ji and Kai Hu and Jinfeng Bai and Xieping Gao},
  title = {{Explicit Relational Reasoning Network for Scene Text Detection}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2025},
  url = {https://openreview.net/forum?id=006TbIzeMJ},
}
Figure