Abstract
Chinese characters are complex structures built from specific stroke sequences. Extracting these strokes is crucial for character recognition and calligraphy evaluation but remains challenging due to overlapping strokes and stylistic variations. This paper proposes a three-stage deep learning framework using Deep Structure Deformable Image Registration. By utilizing a standard character template as prior knowledge, the system achieves state-of-the-art accuracy in stroke separation and matching across diverse handwriting and calligraphy styles.
Introduction
Existing stroke extraction methods typically rely on morphological features, which often fail when strokes intersect or when stylistic variations obscure the underlying structure. Computers lack the inherent semantic knowledge that a human uses to identify strokes—for instance, knowing that a horizontal stroke should not abruptly transition into a vertical one. To address these challenges, researchers from the Chinese Academy of Sciences introduced a novel framework presented at AAAI-23. Their approach leverages a reference font (Kaiti) as "prior knowledge" to guide the extraction process even in complex handwriting or stylized calligraphy.
Methodology
The proposed pipeline consists of three specialized networks designed to progressively refine the stroke extraction process.
- SDNet (Structure Deformable Image Registration Network): This core component performs image registration by warping a reference template to match the target character. Unlike standard registration models that may distort local features, SDNet employs a local linear stroke spatial transformation. This ensures that while the overall character structure deforms to match the target layout, individual strokes maintain their linear and stable morphology.
- SegNet (Image Semantic Segmentation Network): Guided by the output from SDNet, SegNet segments the target character into seven primary stroke categories. This semantic step helps the model distinguish between different stroke types that may be physically touching or overlapping.
- ExtractNet (Single Stroke Extraction Network): The final stage integrates the target image, registered prior information, and semantic features to extract each individual stroke with high precision, following the standard stroke order.
Datasets
The framework was evaluated using two newly constructed datasets:
- CCSEDB (Chinese Calligraphy Character Stroke Extraction Dataset): Contains 5,000 records across various calligraphy styles.
- RHSEDB (Regular Handwriting Character Stroke Extraction Dataset): Includes 28,080 handwriting samples from 17 different writers, derived from the CASIA-OLHWDB dataset. The standard Kaiti font served as the reference template for all registration tasks.
Results and Discussion
The proposed method significantly outperformed existing baselines such as PathNet and PathMatchNet in both accuracy and structural integrity.
On the CCSEDB calligraphy dataset, the method achieved a mean Intersection over Union (mIOU) of 0.924, a substantial improvement over PathMatchNet's 0.692. Furthermore, SDNet demonstrated superior registration quality by maintaining stroke integrity better than standard deformable models like VoxelMorph. The system also achieved near 100% accuracy in matching extracted strokes to their corresponding names in the reference template.
Conclusion
By treating stroke extraction as a registration problem guided by structural priors, this research provides a robust solution for analyzing complex Chinese characters. While the method excels in most scenarios, challenges remain for extremely messy or cursive "scribbled" strokes where morphology is indistinguishable. Future work will focus on enhancing registration robustness to handle even more irregular handwriting styles.
Code is available at: https://github.com/MengLi-l1/StrokeExtraction
Summary updated