Abstract
In recent years, attention-based scene text recognition methods have become the standard due to their ability to adaptively focus on specific image regions during decoding. However, these methods often produce an attention matrix with a nearly one-hot distribution, meaning the model focuses on a tiny area or even a single point. Despite this, traditional models still weight and sum the entire feature map, leading to significant computational redundancy. This paper introduces the Single-Point Decoding Network (SPDN), an efficient attention-free alternative. By utilizing a Single-Point Sampling Module (SPSM), the network identifies and samples exactly one key point per character. Extensive experiments demonstrate that SPDN significantly improves decoding efficiency and speed without sacrificing recognition performance.
Introduction
Scene Text Recognition (STR)—the task of reading text from street signs, packaging, and natural environments—is a cornerstone of computer vision. While attention-based decoders have driven state-of-the-art results, they are computationally expensive. The authors of SPDN identified a critical inefficiency: the "attention" these models pay to an image is often so concentrated that calculating weights for every pixel is mathematically redundant.
SPDN proposes a shift in paradigm: instead of calculating a global attention map and then narrowing it down, the model should directly predict the most relevant spatial point for each character. This approach reduces the computational footprint (FLOPs) and memory usage, making high-performance STR more viable for real-time applications.
The SPDN Architecture
The SPDN framework consists of three primary stages designed to transform a raw image into a character sequence efficiently:
1. Rectification
To handle the challenges of curved or perspective-distorted text, the model employs a Rectifier based on a Thin Plate Spline (TPS) network. This stage "straightens" the input image, providing a normalized view for the subsequent encoder.
2. Feature Extraction (Encoder)
A standard Convolutional Neural Network (CNN) backbone serves as the encoder. It extracts two-dimensional visual features from the rectified image, creating a rich representation of the text's geometry and appearance.
3. Single-Point Decoding (The Core Innovation)
The traditional attention decoder is replaced by the Single-Point Sampling Module (SPSM). The SPSM utilizes a Position Alignment Unit (PAU) to predict the $(x, y)$ coordinates of the next character's key point. This prediction is informed by the hidden state of an LSTM and the position of the previously decoded character.
Once the coordinate is predicted, a grid sampler extracts the feature at that specific point. To ensure the model learns a logical, left-to-right (or sequence-appropriate) progression, the authors introduced a Position Constraint Loss ($L_{dist}$). This loss function penalizes erratic jumps between sampling points, forcing the model to maintain spatial consistency.
Experimental Results
Efficiency and Speed
SPDN's most striking advantage is its efficiency. By bypassing the dense calculations of attention maps, the model achieves:
- Inference Speed: 0.65 ms per image, making it nearly 2x faster than the Decoupled Attention Network (DAN) and 16x faster than NRTR.
- Computational Load: Only 1.04 GFLOPs, positioning it as one of the most lightweight high-performance models available.
Accuracy Across Benchmarks
The model was trained on the Synth90K and SynthText datasets and evaluated across six major benchmarks. SPDN consistently outperformed its baseline (ASTER), with notable improvements in "irregular" text categories:
- ICDAR 2015: 1.8% improvement.
- CUTE80: 2.1% improvement.
The researchers also experimented with the number of sampling points. They found that using a single point provided the optimal balance; increasing the points to two or four resulted in negligible accuracy gains while significantly increasing the computational cost.
Insights on Sampling Strategy
The study compared Serial and Parallel sampling. In parallel sampling, the model attempts to predict all character points simultaneously. In serial sampling, it predicts them one by one. The results showed that serial sampling is superior because it allows the model to use the "prior" knowledge of the previous character's location to find the next one, mimicking the way humans scan text.
Conclusion and Future Work
SPDN proves that attention is often "overkill" for scene text recognition. By forcing the encoder to pack all necessary information for a character into a single spatial point, the model becomes more discriminative and significantly faster.
While SPDN excels in efficiency, there remains a performance gap compared to "heavyweight" models like ABINet, likely due to SPDN's use of a simpler backbone and LSTM. Future research will focus on integrating more powerful backbones (like ResNet with Feature Pyramid Networks) and stronger language modeling components to improve performance on blurry or occluded text. Additionally, the single-point decoding concept may hold promise for other sequence-to-sequence tasks in computer vision.
Summary updated