Scene Text Detection Based on Two-Branch Feature Extraction

Mayire Ibrayim, Yuan Li, Askar HamdullaSensors 2022 · DOI

TL;DR

This paper introduces a novel scene text detection method that enhances feature extraction using a "Residual Correction Branch" (RCB) to capture broader contextual information and a "Two-Branch Attention Feature Fusion" (TB-AFF) module to effectively integrate multi-scale features. This approach aims to improve the accuracy and robustness of text detection in challenging natural scenes, outperforming existing methods on several benchmark datasets.

What Problem Does This Paper Address?

Detecting text in natural scene images is a notoriously difficult task. Unlike structured text in documents, scene text can appear in various orientations (horizontal, skewed, curved), with diverse fonts, sizes, and colors, often against complex and cluttered backgrounds. Existing deep learning methods, particularly those based on convolutional neural networks (CNNs), struggle with this due to limitations in their receptive fields, which hinder their ability to capture global semantic context. This can lead to missed detections or false positives, especially for long or irregularly shaped text. Furthermore, while feature pyramid networks (FPNs) help fuse multi-scale features, higher-level semantic information for smaller text instances can be lost.

Methods

The proposed method builds upon the DBNet architecture, a segmentation-based approach known for its efficiency. The core innovations are:

  • Residual Correction Branch (RCB): This module is designed to enhance the feature extraction capabilities of the backbone network (e.g., ResNet) without altering its architecture. It employs a parallel, dual-branch structure. One branch retains spatial information through a standard convolution, while the other uses average pooling to increase the receptive field and then upsampling to restore spatial resolution. By combining these, RCB captures richer contextual information around each pixel, improving the network's ability to discern text from complex backgrounds. It's noted for being lightweight and easily embeddable.
  • Two-Branch Attention Feature Fusion (TB-AFF) Module: This module is integrated into the FPN structure to improve feature fusion. It addresses the issue of information loss in deeper layers of FPNs. TB-AFF uses two branches: a global feature branch that captures overall context (similar to SENet but adapted for text) and a local feature branch that preserves fine-grained details. By combining global and local attention mechanisms, it effectively fuses multi-scale features, enhancing the representation of both large and small text instances and improving localization accuracy.
  • Differentiable Binarization: The paper utilizes an adaptive, differentiable binarization method to convert the predicted probability map into a text region. This approach, integrated into the training process, helps overcome the non-differentiable nature of standard binarization, leading to better optimization and more accurate text detection.

Datasets

The proposed method was evaluated on three widely used public datasets:

  • ICDAR2015: A dataset for multi-directional text detection.
  • Total-Text: Specifically designed for detecting curved text.
  • MSRA-TD500: A dataset featuring multi-language and multi-category text.

The authors also mention pre-training on the SynthText dataset.

Key Insights and Ideas

  • Context is King (for Text): The RCB module highlights that simply increasing network depth isn't always the answer. Capturing relevant contextual information around text pixels, even with a lightweight module, significantly boosts detection performance.
  • Attention for Text Needs Nuance: While global attention is useful, it can overlook smaller text instances. The TB-AFF module's dual-branch approach (global + local) is crucial for effectively handling the diverse scales of text found in natural scenes.
  • Feature Fusion is More Than Just Merging: The TB-AFF module demonstrates that intelligently fusing features from different scales and semantic levels, guided by attention, is key to robust text detection.
  • Lightweight Modules Can Be Powerful: Both RCB and TB-AFF are designed to be computationally efficient, showing that significant performance gains can be achieved without drastically increasing model complexity.

Key Results

The proposed method, combining RCB and TB-AFF with a ResNet backbone, demonstrated superior performance across the tested datasets:

  • ICDAR2015: Achieved an F-score of 83.19% (with ResNet-18), surpassing the original DBNet by 2.39%.
  • Total-Text: Achieved an F-score of 82.95% (with ResNet-18), outperforming the original DBNet by 2.15%.
  • MSRA-TD500: Achieved an F-score of 85.61% (with ResNet-18), showing significant improvements over the original DBNet.

Ablation studies confirmed that both RCB and TB-AFF individually contributed to performance gains, but their combination yielded the best results, indicating their synergistic effect. The method also showed robustness in handling various text types, including multi-directional, multi-lingual, and curved texts.

Limitations and Future Work

  • Architecture Optimization: The authors acknowledge that exploring the absolute best architecture is beyond the scope of this paper. They suggest that further research could focus on designing even more effective structures.
  • Complexity Reduction: Future work aims to further optimize the segmentation network to reduce model complexity, shorten training times, and improve overall performance and accuracy.
  • Generalization: While the paper demonstrates good universality, further exploration of its applicability to even more diverse and challenging scenarios could be beneficial.

Summary updated

BibTeX
@article{ibrayim2022scene,
  author = {Mayire Ibrayim and Yuan Li and Askar Hamdulla},
  title = {{Scene Text Detection Based on Two-Branch Feature Extraction}},
  journal = {Sensors},
  year = {2022},
  doi = {10.3390/s22166262},
}

Loading related content...

Figure