Convolutional Regression Network for Multi-Oriented Text Detection

Junyu Gao, Qi Wang, Yuan Yuan

Detecting text in natural scenes—on street signs, product packaging, or billboards—is significantly more complex than scanning a document. Text in these environments varies wildly in scale, lighting, and, most importantly, orientation. Traditional object detection frameworks like Faster R-CNN or SSD often rely on horizontal "anchor boxes." While effective for standard objects, these boxes are a poor fit for tilted or vertical text, often capturing excessive background noise that confuses the classifier.

The Convolutional Regression Network (CRN) addresses this by moving away from fixed anchors toward an end-to-end trainable system that "thinks" in rotations from the very start.

The CRN Architecture

The CRN architecture is a multi-task framework designed to handle the complexities of scene text through four primary stages:

  1. Backbone: A standard VGG-Net extracts basic features from the input image.
  2. Hierarchical Deconvolution Module (HDM): This module aggregates features from different layers of the network to handle text of various sizes. Crucially, it incorporates a bidirectional LSTM to capture long-range contextual information, helping the network distinguish between actual text and text-like textures (like fences or bricks).
  3. Text-line and Geometry Segmentation Module (TGM): Instead of guessing boxes, this module performs pixel-wise segmentation. It produces a "text-line map" (identifying text regions) and a "geometry map" (predicting the distance to the four boundaries of the text box and the rotation angle).
  4. Classification and Regression Module (CRM): This refinement stage takes the rough proposals from the segmentation step and applies a Rotated ROI Pooling layer. This layer rotates the feature maps to a horizontal orientation based on the predicted angle, allowing standard fully connected layers to precisely refine the text's location and classification.
The CRN architecture featuring the Hierarchical Deconvolution Module and Rotated ROI Pooling.

Key Insights and Innovations

Segmentation as a Proposal Generator

A core innovation of CRN is using segmentation masks to generate proposals rather than fixed anchors. This allows the network to adapt to the actual shape and tilt of the text before the refinement stage begins. To prevent the network from merging adjacent lines of text, the authors utilized a "shrink trick," training the model on ground truth masks that are slightly smaller than the actual text line. By targeting the core of the text, the model maintains better separation.

Contextual Awareness

By integrating a Bi-LSTM in the deconvolution stage, the model learns that text usually appears in sequences. This spatial context significantly reduces false positives in complex visual environments where repetitive patterns might otherwise be mistaken for characters.

Rotated ROI Pooling

This is the technical "secret sauce." By transforming a tilted region into a normalized horizontal feature map, the network can use traditional, highly-optimized regression techniques to "polish" the final bounding box, ensuring high precision even for extreme angles.

Visualizing the pipeline: input images (left), text-line segmentation maps (center), and final multi-oriented detections (right).

Performance and Benchmarks

The CRN was evaluated on three mainstream benchmarks: ICDAR2013 (horizontal text), ICDAR2015 (incidental scene text), and MSRA-TD500 (multi-lingual, multi-oriented text).

  • Precision: On ICDAR2015, the CRN achieved an F-measure of 79.4%, outperforming several contemporary models like SegLink and CTPN.
  • Efficiency: The model is remarkably fast, with an inference time of 80ms per image. This makes it competitive with high-speed detectors like EAST and significantly faster than traditional multi-step pipelines.
  • Robustness: Qualitative results demonstrate the model's ability to successfully localize text on curved surfaces and in low-contrast environments where traditional methods often fail.

Limitations and Future Work

While the model performs excellently on ICDAR datasets, its performance on MSRA-TD500 was slightly lower (72% F-measure). The authors attribute this to the relatively small size of the MSRA training set, which makes it difficult for a deep CNN to generalize across the vast diversity of Chinese and English fonts. Future work may involve exploring synthetic data generation to bolster performance on multi-lingual datasets. Additionally, while CRN handles multi-oriented rectangles well, extending the architecture to handle curved text remains an open challenge.

Summary updated

Abstract

Multi-oriented text detection in the wild is a challenging task due to the variations of scales, orientations, illumination, and languages. The traditional anchor mechanism on generic object detection can only generate horizontal proposals, which cannot be applied to detecting multi-oriented text regions. Considering this, in this paper, we propose a novel convolutional regression network (CRN) to localize multi-oriented text in natural images, which consists of two components: region proposal extractor and text locator. To be specific, we first present a hierarchical deconvolution module (HDM), a text-line and geometry segmentation module (TGM) to segment the multi-oriented proposals accurately, both of which are fully convolutional networks. Then, a classification and regression module (CRM) is adopted to process the proposals and obtain the final localization results. The whole framework can be trained in an end-to-end mechanism which is suitable for detecting multi-oriented texts. The extensive experiments are conducted on three mainstream scene-text datasets, and the experimental results evidence the proposed CRN achieves competitive performance.
BibTeX
@article{gao2019convolutional,
  author = {Junyu Gao and Qi Wang and Yuan Yuan},
  title = {{Convolutional Regression Network for Multi-Oriented Text Detection}},
  journal = {IEEE Access},
  year = {2019},
  doi = {10.1109/access.2019.2929819},
}
Figure