Abstract: Background elimination for noisy character images or character images from real scenes is still a challenging problem, due to the bewildering backgrounds, uneven illumination, low resolution and different distortions. We propose a stroke-based character reconstruction (SCR) method that uses a weighted quadratic Bezier curve (WQBC) to represent strokes of a character. Only training on our synthetic data, our stroke extractor can achieve excellent reconstruction effect in real scenes. Meanwhile, it can also help achieve great ability in defending adversarial attacks of character recognizers.
Introduction
Character recognition in "the wild"—encompassing street signs, blurry documents, or low-resolution captures—remains a significant challenge for computer vision. Standard Deep Neural Networks (DNNs) are highly effective at classification but often fail to capture the underlying structural information of a character. Because they treat images primarily as grids of pixels, they are easily confused by uneven illumination, complex backgrounds, and "adversarial attacks"—subtle pixel perturbations designed to fool AI models.
The SCR framework addresses this by introducing structural priors into the recognition process. By conceptualizing a character as a sequence of strokes rather than a collection of pixels, the system naturally filters out non-structural noise, creating a model that is both more interpretable and more resilient.
Methodology
The SCR framework utilizes an encoder-decoder architecture designed to map noisy inputs to geometric parameters:
- Stroke Representation: Instead of pixels, the model uses Weighted Quadratic Bezier Curves (WQBC). Each stroke is defined by three points $(x, y)$, with each point assigned a weight $w$ representing the stroke's thickness. A single character is typically represented by four such curves (totaling 36 parameters).
- Stroke Extractor (Encoder): A VGG-like network takes a noisy input image and predicts the 36 parameters required to define the strokes.
- Differentiable Decoder: To allow for end-to-end training via backpropagation, the authors pre-trained a neural network to simulate the rendering process. This "Differentiable Decoder" enables the system to optimize stroke parameters directly against the visual output.
- Training Strategy: The model is trained primarily on synthetic data—digitally generated characters with randomized noise and distortions—to minimize the $L_2$ distance between the reconstructed clean image and the ground truth.
Experimental Results
The authors evaluated their model on three primary datasets: MNIST (handwritten digits), SVHN (Street View House Numbers), and a custom Synthetic SVHN dataset.
Adversarial Defense
A key insight of the SCR method is the "Structural Bottleneck." By forcing the image through a bottleneck of stroke parameters, the model acts as a powerful filter. Adversarial noise cannot be represented as Bezier curves, so the decoder effectively "purifies" the image. On the MNIST dataset, SCR maintained 94% accuracy against FGSM attacks, significantly outperforming standard models which dropped to 54%. It showed similar robustness against powerful Carlini-Wagner ($L_\infty, L_2,$ and $L_0$) attacks.
Real-World Performance
The model achieved an 89% reconstruction success rate on real SVHN images despite being trained only on synthetic data. This suggests that the "language of strokes" is universal enough to bridge the gap between synthetic fonts and real-world handwriting or printing.
Limitations and Future Work
While effective for digits and Latin alphabets, the current model uses a fixed number of strokes (four). This would likely struggle with more complex scripts like Chinese characters, which can have dozens of strokes. Future directions include integrating this stroke-based approach into sequence-based character recognizers (like RNNs) to treat predicted stroke parameters as time-series data, further improving accuracy and versatility.
Summary updated