Stroke Extraction of Chinese Character Based on Deep Structure Deformable Image Registration

Meng Li, Yahan Yu, Yi Yang, Guanghao Ren, Jian WangAAAI 2023 · DOI

Abstract

Chinese characters are complex structures built from specific stroke sequences. Extracting these strokes is crucial for character recognition and calligraphy evaluation but remains challenging due to overlapping strokes and stylistic variations. This paper proposes a three-stage deep learning framework using Deep Structure Deformable Image Registration. By utilizing a standard character template as prior knowledge, the system achieves state-of-the-art accuracy in stroke separation and matching across diverse handwriting and calligraphy styles.

Introduction

Existing stroke extraction methods typically rely on morphological features, which often fail when strokes intersect or when stylistic variations obscure the underlying structure. Computers lack the inherent semantic knowledge that a human uses to identify strokes—for instance, knowing that a horizontal stroke should not abruptly transition into a vertical one. To address these challenges, researchers from the Chinese Academy of Sciences introduced a novel framework presented at AAAI-23. Their approach leverages a reference font (Kaiti) as "prior knowledge" to guide the extraction process even in complex handwriting or stylized calligraphy.

Methodology

The proposed pipeline consists of three specialized networks designed to progressively refine the stroke extraction process.

Overview of the three-stage pipeline including SDNet, SegNet, and ExtractNet.
  1. SDNet (Structure Deformable Image Registration Network): This core component performs image registration by warping a reference template to match the target character. Unlike standard registration models that may distort local features, SDNet employs a local linear stroke spatial transformation. This ensures that while the overall character structure deforms to match the target layout, individual strokes maintain their linear and stable morphology.
  2. SegNet (Image Semantic Segmentation Network): Guided by the output from SDNet, SegNet segments the target character into seven primary stroke categories. This semantic step helps the model distinguish between different stroke types that may be physically touching or overlapping.
  3. ExtractNet (Single Stroke Extraction Network): The final stage integrates the target image, registered prior information, and semantic features to extract each individual stroke with high precision, following the standard stroke order.

Datasets

The framework was evaluated using two newly constructed datasets:

  • CCSEDB (Chinese Calligraphy Character Stroke Extraction Dataset): Contains 5,000 records across various calligraphy styles.
  • RHSEDB (Regular Handwriting Character Stroke Extraction Dataset): Includes 28,080 handwriting samples from 17 different writers, derived from the CASIA-OLHWDB dataset. The standard Kaiti font served as the reference template for all registration tasks.

Results and Discussion

The proposed method significantly outperformed existing baselines such as PathNet and PathMatchNet in both accuracy and structural integrity.

Comparison of stroke extraction results between the proposed method and baseline models PathNet and PathMatchNet.

On the CCSEDB calligraphy dataset, the method achieved a mean Intersection over Union (mIOU) of 0.924, a substantial improvement over PathMatchNet's 0.692. Furthermore, SDNet demonstrated superior registration quality by maintaining stroke integrity better than standard deformable models like VoxelMorph. The system also achieved near 100% accuracy in matching extracted strokes to their corresponding names in the reference template.

Conclusion

By treating stroke extraction as a registration problem guided by structural priors, this research provides a robust solution for analyzing complex Chinese characters. While the method excels in most scenarios, challenges remain for extremely messy or cursive "scribbled" strokes where morphology is indistinguishable. Future work will focus on enhancing registration robustness to handle even more irregular handwriting styles.

Code is available at: https://github.com/MengLi-l1/StrokeExtraction

Summary updated

BibTeX
@inproceedings{li2023stroke,
  author = {Meng Li and Yahan Yu and Yi Yang and Guanghao Ren and Jian Wang},
  title = {{Stroke Extraction of Chinese Character Based on Deep Structure Deformable Image Registration}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2023},
  doi = {10.1609/aaai.v37i1.25220},
}
Figure