BPDO: Boundary Points Dynamic Optimization for Arbitrary Shape Scene Text Detection

Jinzhi Zheng, Libo Zhang, Yanjun Wu, Chen Zhao

Introduction

Detecting text in natural scenes—such as street signs, product packaging, or curved advertisements—presents a significant challenge compared to standard document scanning. Text in the "wild" is often arbitrary-shaped, appearing curved, slanted, or irregularly distorted. Current research typically follows two paths: segmentation-based methods, which are flexible but noise-sensitive, and boundary-point-based methods, which often rely on global modeling that neglects crucial local spatial details.

To address these limitations, we present BPDO (Boundary Points Dynamic Optimization), a two-stage framework designed for high-precision detection of arbitrary-shaped text. The core philosophy of BPDO is that the exact edge of a word is best determined by examining the immediate neighborhood of boundary points rather than the text instance as a whole.

The BPDO Framework

The BPDO architecture operates in two distinct stages to move from a coarse text proposal to a refined, accurate boundary.

Stage 1: Text Boundary Points Proposal

The process begins with a ResNet-50 backbone and Feature Pyramid Networks (FPN) to extract multi-scale visual features. These features are processed by a novel Text Aware Module (TAM). The TAM employs both channel and spatial attention mechanisms to fuse features across different scales, allowing the model to effectively "sense" text regions regardless of their orientation or size. This stage generates a coarse set of boundary points along with auxiliary maps for distance, direction, and classification.

Stage 2: Boundary Points Dynamic Optimization (DOM)

The defining innovation of BPDO is the Dynamic Optimization Module (DOM). Inspired by deformable attention, the DOM iteratively refines the coarse boundary points. Instead of a global view, the module samples features from the immediate local neighborhood of each point. By querying these local features, the DOM calculates precise offsets to nudge each point toward the actual text edge. Empirical results suggest that three iterations are sufficient to achieve high-precision alignment.

The BPDO framework overview, illustrating the two-stage process: Text Boundary Points Proposal (TAM) and Boundary Points Dynamic Optimization (DOM).

Experimental Results

BPDO was rigorously evaluated on several benchmark datasets, including Total-Text (curved text), CTW1500 (long curved text), and MSRA-TD500 (multi-directional English and Chinese text).

Performance Highlights

  • MSRA-TD500: BPDO achieved a state-of-the-art (SOTA) F-measure of 91.47%, outperforming the previous best model (TextBPN++) by 1.3%.
  • CTW1500 & Total-Text: The model reached F-measures of 86.56% and 87.2% respectively, demonstrating robust performance across diverse text geometries.

Ablation studies confirmed the synergy between the TAM and DOM; while the DOM improved the baseline F-measure by 0.46%, the combined architecture yielded a total improvement of 1.37%.

Qualitative detection results of BPDO on MSRA-TD500, CTW1500, and Total-Text datasets, demonstrating robust performance on multi-directional and curved text.

Conclusion and Future Work

BPDO demonstrates that local spatial features are more informative for boundary refinement than global relationships in scene text detection. By focusing on the area immediately surrounding boundary points, the model avoids distraction from distant image features and achieves superior accuracy.

While BPDO sets a new standard for detection, it currently functions as a standalone detector. Future research will focus on evolving this framework into an end-to-end scene text spotting system that integrates both detection and recognition into a single, unified pipeline.

Summary updated

Abstract

Arbitrary shape scene text detection is of great importance in scene understanding tasks. Due to the complexity and diversity of text in natural scenes, existing scene text algorithms have limited accuracy for detecting arbitrary shape text. In this paper, we propose a novel arbitrary shape scene text detector through boundary points dynamic optimization(BPDO). The proposed model is designed with a text aware module (TAM) and a boundary point dynamic optimization module (DOM). Specifically, the model designs a text aware module based on segmentation to obtain boundary points describing the central region of the text by extracting a priori information about the text region. Then, based on the idea of deformable attention, it proposes a dynamic optimization model for boundary points, which gradually optimizes the exact position of the boundary points based on the information of the adjacent region of each boundary point. Experiments on CTW-1500, Total-Text, and MSRATD500 datasets show that the model proposed in this paper achieves a performance that is better than or comparable to the state-of-the-art algorithm, proving the effectiveness of the model.
BibTeX
@inproceedings{zheng2024bpdo,
  author = {Jinzhi Zheng and Libo Zhang and Yanjun Wu and Chen Zhao},
  title = {{BPDO: Boundary Points Dynamic Optimization for Arbitrary Shape Scene Text Detection}},
  booktitle = {IEEE International Conference on Acoustics, Speech and Signal Processing},
  year = {2024},
  url = {https://openreview.net/forum?id=YovX4nTgU2},
}
Figure