Abstract
This paper presents a purely convolutional neural network designed for handwritten string recognition. By integrating a novel Flexible Spatial Pyramid Pooling (FSPP) layer and a dual-module output structure—comprising Counting and Prediction Modules—the system effectively handles arbitrary input widths and produces variable-length label sequences. This approach bypasses the need for complex character segmentation or recurrent architectures, achieving state-of-the-art results on benchmark digit string datasets.
Introduction
Recognizing handwritten strings, such as those found on bank checks or postal envelopes, is a significant challenge due to character connectivity. When characters touch or overlap, traditional segmentation-based methods often fail. While Recurrent Neural Networks (RNNs) with Connectionist Temporal Classification (CTC) have become popular for sequence tasks, they can be difficult to train and require substantial data. This research explores a segmentation-free, CNN-based sequence labeling framework that treats the entire string as a single context.
Proposed Architecture
The authors propose a "Sequence Labeling Convolutional Network" that leverages a deep convolutional backbone to extract robust features from images of arbitrary sizes.
As shown in the architecture diagram, the network utilizes an Inception-style backbone to capture multi-scale local features. The output is divided into two specialized components:
- Counting Module (CM): Predicts the total number of characters ($N$) in the string.
- Prediction Module (PM): Consists of $K$ parallel softmax classifiers (where $K$ is the maximum supported string length). The CM informs the system which PM outputs are relevant for the final predicted sequence.
Flexible Spatial Pyramid Pooling (FSPP)
A core innovation of this work is the Flexible Spatial Pyramid Pooling (FSPP) layer. Standard Spatial Pyramid Pooling (SPP) allows networks to accept various input sizes by pooling features into a fixed number of spatial bins. However, for text recognition, the horizontal dimension is more critical for capturing sequential information than the vertical dimension.
FSPP improves upon standard SPP by fixing the vertical scale while dynamically adjusting the horizontal scale based on the image's aspect ratio. This ensures that longer strings receive more horizontal bins, preserving the necessary spatial resolution to distinguish individual characters in a sequence.
Experimental Results
The model was evaluated on several prominent benchmarks, including CVL, ORAND-CAR (bank check digits), and a new dataset, PhPAIS (11-digit phone numbers).
- State-of-the-Art Performance: The network outperformed existing methods in the ICFHR2014 Handwritten Digit String Recognition competition, achieving a mean accuracy of 81.72% across CVL and ORAND-CAR. On the particularly difficult CAR-B dataset, it reached 83.32% accuracy.
- FSPP Efficacy: On the long-string PhPAIS dataset, FSPP demonstrated superior stability. While standard SPP saw a 2% drop in accuracy when reducing scales, FSPP maintained performance with a negligible 0.16% drop, proving its necessity for handling varying sequence lengths.
Conclusion
This research demonstrates that recurrence is not strictly necessary for sequence labeling in vision tasks. A well-designed CNN with a flexible pooling layer and multi-head output can achieve superior results in handwritten string recognition. While the architecture currently requires a predefined maximum length ($K$), it offers a robust, segmentation-free alternative that is highly effective for real-world, noisy digit strings.
Summary updated