Introduction
Scene Text Recognition (STR) has achieved high accuracy for regular, horizontal text. However, irregular text—characterized by curvature, perspective distortion, or multi-orientation—remains a significant challenge. Traditional approaches often rely on Spatial Transformer Networks (STN) using Thin Plate Splines (TPS) to "unwarp" text. However, these methods can suffer from "over-rectification," where excessive flexibility leads to character distortion or the loss of edge information.
This paper introduces Rectifying Principle Irregularities (RPI), a framework designed to perform "rough" but robust rectification. By fitting text to predefined quadratic Bézier curves, RPI preserves vital visual features, allowing modern neural recognizers to handle minor remaining irregularities.
The RPI Framework
The RPI method is composed of four integrated modules:
- Densities and Orientations Estimating Module (DOEM): A U-Net-style network that predicts pixel-level character density and local text orientation.
- Curve Fitting Module (CFM): Instead of complex optimization, RPI selects the best fit from 37 predefined quadratic Bézier curves. Selection is based on alignment with the DOEM's density and orientation maps.
- Grid Sampling Module (GSM): This module generates a sampling grid to flatten the image. It employs a "boundary-aware" strategy to ensure that characters near the image edges are not clipped during rectification.
- Sequence Recognizing Module (SRM): A CNN-LSTM-Attention encoder-decoder architecture that performs final character recognition on the rectified image.
Key Insights
A core philosophy of RPI is the "Good Enough" Principle. The authors observed that modern recognizers are inherently robust to minor rotations (up to ±20°) and scale variations. Consequently, rectification does not need to be mathematically perfect; it only needs to remove the "principal" irregularity to bring the text into a roughly horizontal alignment. By choosing from a discrete set of candidate curves rather than continuous optimization, the model avoids "hallucinated" distortions and remains computationally efficient.
Experimental Results
The RPI model was evaluated on seven major benchmarks, including regular text (IIIT5K, SVT, IC03, IC13) and irregular text (IC15, SVTP, CUTE80).
- State-of-the-Art Performance: RPI achieved 91.7% accuracy on the CUTE80 dataset, a benchmark specifically for curved text, outperforming the previous best method (SAR) by 2.1%.
- General Robustness: It consistently outperformed other rectification-based models like ASTER and RARE. On the IIIT5K dataset, it reached 95.1% accuracy.
- Efficiency: The CFM module is parameter-free and uses a fixed set of curves, making the system fast and easy to implement.
Conclusion and Future Work
While RPI excels at curved text, it shows slightly lower dominance on extreme perspective distortions when trained solely on synthetic data. Future research aims to make the framework entirely end-to-end and apply the "principal irregularity" approach to full-page text spotting. By focusing on preserving information rather than achieving perfect geometric flatness, RPI provides a more robust path for recognizing text in complex real-world scenes.
Summary updated