Abstract
Background elimination for noisy character images or character images from real scene is still a challenging problem, due to the bewildering backgrounds, uneven illumination, low resolution and different distortions. We propose a stroke-based character reconstruction (SCR) method that use a weighted quadratic Bezier curve (WQBC) to represent strokes of a character. Only training on our synthetic data, our stroke extractor can achieve excellent reconstruction effect in real scenes. Meanwhile, it can also help achieve great ability in defending adversarial attacks of character recognizers.
Introduction
Character recognition in natural environments, such as street numbers or signage, remains a significant challenge for computer vision. Traditional Deep Neural Networks (DNNs) often struggle with uneven lighting, low resolution, and complex backgrounds. Furthermore, these models are notoriously vulnerable to adversarial attacks—subtle pixel-level perturbations that can lead to catastrophic misclassification despite being invisible to the human eye.
The core issue is that standard models treat images as collections of independent pixels rather than structured sequences of strokes. By shifting the focus to structural information, we can develop systems that are inherently more robust to noise and intentional interference.
Methodology: Stroke-based Character Reconstruction (SCR)
The proposed SCR method utilizes an encoder-decoder architecture designed with a structural bottleneck. Instead of reconstructing pixels directly, the model represents characters as a set of Weighted Quadratic Bezier Curves (WQBC).
Stroke Representation
Each stroke is defined by nine parameters: three $(x, y)$ coordinate points and their associated thicknesses ($w$). This compact representation forces the model to capture the essential geometry of the character.
The Pipeline
- Stroke Extractor (Encoder): A VGG-like neural network processes the input image to predict parameters for a fixed number of strokes (e.g., four curves per character).
- Differentiable Decoder: A pre-trained, differentiable decoder converts these parameters back into a clean image. This allows the entire pipeline to be trained via backpropagation using an $L_2$ loss between the reconstructed image and a clean ground truth.
By forcing the output through this "stroke bottleneck," the system acts as a structural low-pass filter. Since the decoder can only draw smooth curves, it naturally ignores pixel-level adversarial noise.
Experimental Setup
The model was evaluated on several datasets:
- MNIST: Used primarily to test resilience against adversarial attacks.
- SVHN (Street View House Numbers): A challenging real-world dataset of house numbers.
- Synthetic Data: To overcome the lack of labeled stroke data, the authors generated 200,000 synthetic images using 250 fonts and various augmentations (rotation, blur, and noise).
Key Results
Adversarial Defense
The SCR method demonstrates remarkable robustness. On the MNIST dataset, it achieved an average accuracy of 88.60% across various adversarial attacks, including FGSM, BIM, and JSMA. In contrast, undefended models dropped to 13% accuracy. This performance is nearly on par with human performance (88.90%).
Real-World Generalization
Despite being trained exclusively on synthetic data, the model achieved an 89% reconstruction success rate on the SVHN test set. This "zero-shot" generalization suggests that the "language of strokes" is a more universal and robust feature set than raw pixel patterns.
Conclusion and Future Work
The SCR method provides a powerful defense against adversarial attacks and background noise by leveraging structural priors. While highly effective for digits and simple characters, future work will explore scaling this approach to more complex scripts (like Chinese) and implementing dynamic stroke counts using Reinforcement Learning to handle varying character complexity.
Reference: Huang, Z., Heng, W., Tao, Y., & Zhou, S. (2018). Stroke-based Character Reconstruction. arXiv:1806.08990v3 [cs.CV].
Summary updated