DP-Font: Chinese Calligraphy Font Generation Using Diffusion Model and Physical Information Neural Network

Liguo Zhang, Yalong Zhu, Achref Benarab, Yusen Ma, Yuxin Dong, Jianguo Sun

Abstract

As a typical visual art form, Chinese calligraphy has a long history and aesthetic value. However, current methods for generating Chinese fonts still struggle with complex character shapes and lack personalized writing styles. To address these issues, we propose a font generation method for Chinese Calligraphy based on a diffusion model incorporating a physical information neural network (PINN), named DP-Font. Firstly, multi-attribute guidance is combined to guide the generation process of the diffusion model and introduce the critical constraint of stroke order in Chinese characters, aiming to significantly improve the quality of the generated results. We then incorporate physical constraints into the neural network loss function, utilizing physical equations to provide in-depth guidance and constraints on the learning process. By learning the movement rule of the nib and the diffusion pattern of the ink, DP-Font can generate personalized calligraphy styles. The generated fonts are very close to the calligraphers' works. Compared with existing deep learning-based techniques, DP-Font has made significant progress in enhancing the physical plausibility of the model, generating more realistic and high-quality results.

Introduction

Generating Chinese calligraphy is a notoriously difficult task for artificial intelligence. Unlike standard digital fonts, calligraphy involves complex character shapes, personalized artistic styles, and the subtle physical behavior of ink on paper. Current state-of-the-art methods, largely based on Generative Adversarial Networks (GANs), often struggle with structural integrity—where complex characters suffer from missing or redundant strokes—and style fidelity. Furthermore, these models often lack physical plausibility, producing strokes that look artificial because they do not account for the mechanics of nib movement or ink diffusion.

DP-Font addresses these challenges by combining the generative power of Denoising Diffusion Probabilistic Models (DDPM) with the structural rigor of stroke order encoding and the physical realism of Physical Information Neural Networks (PINNs).

Methodology

DP-Font introduces a novel framework that conditions the diffusion process on three distinct features: Content (the character identity), Style (the specific calligraphic hand), and Stroke Order.

The DP-Font architecture incorporating multi-attribute guidance and PINN constraints.

Multi-Attribute Guidance and Stroke Encoding

To solve common structural errors, DP-Font explicitly encodes the temporal nature of writing. Each character is represented as a 36-dimensional vector that captures the sequence of the five basic Chinese stroke types: heng (horizontal), shu (vertical), pie (downward left), dian (dot), and zhe (turning). This encoding serves as a structural "blueprint," guiding the diffusion model to place strokes in the correct sequence and spatial position, which is particularly vital for characters with high stroke counts.

Physical Information Neural Network (PINN) Integration

The most significant innovation in DP-Font is the incorporation of a physical diffusion equation into the neural network's loss function. By treating ink concentration $u$ as a variable governed by the heat equation:

$$\frac{\partial u}{\partial t} = D\nabla^2 u$$

the model is forced to minimize a "physics loss." This ensures that generated ink patterns follow the natural behavior of liquid spreading on a porous surface, effectively reducing over-smoothing and artificial artifacts that plague purely data-driven models.

Experimental Results

The model was trained on a dataset of 100 fonts (7,905 characters each) and tested against 10 distinct fonts with intricate structures. DP-Font was compared against five leading models: FUNIT, SC-Font, MX-Font, DG-Font, and the diffusion-based Diff-Font.

Quantitative Performance

DP-Font achieved superior scores across all primary metrics. It demonstrated an 8.48% improvement in Frechet Inception Distance (FID) over the previous best diffusion model, indicating significantly higher visual realism. It also achieved a Structural Similarity Index (SSIM) of 0.735, confirming its ability to maintain the structural essence of the target fonts.

Qualitative Realism

Visual comparisons highlight DP-Font's ability to avoid the "broken" strokes or redundant artifacts common in other models. It successfully captures the distinct "weight" and "flow" of Tang Dynasty masters, such as Gongquan Liu and Zhenqing Yan.

Qualitative comparison of DP-Font against state-of-the-art methods for complex Chinese characters. Red markings indicate structural failures in baseline models.

Conclusion

DP-Font represents a significant step forward in calligraphic font generation by bridging the gap between deep learning and physical laws. By treating stroke order as structural DNA and ink diffusion as a physical constraint, the model produces results that are both aesthetically authentic and structurally sound. While future work remains in synthesizing complex backgrounds and optimizing inference speed, the "physics-meets-diffusion" approach offers a robust template for high-fidelity artistic generation.

Summary updated

Abstract

As a typical visual art form, Chinese calligraphy has a long history and aesthetic value. However, current methods for generating Chinese fonts still struggle with complex character shapes and lack personalized writing styles. To address these issues, we propose a font generation method for Chinese Calligraphy based on diffusion model incorporating physical information neural network (PINN), which is named DP-Font. Firstly, the multi-attribute guidance is combined to guide the generation process of the diffusion model and introduce the critical constraint of stroke order in Chinese characters, aiming to significantly improve the quality of the generated results. We then incorporate physical constraints into the neural network loss function, utilizing physical equations to provide in-depth guidance and constraints on the learning process. By learning the movement rule of the nib and the diffusion pattern of the ink, DP-Font can generate personalized calligraphy styles. The generated fonts are very close to the calligraphers' works. Compared with existing deep learning-based techniques, DP-Font has made significant progress in enhancing the physical plausibility of the model, generating more realistic and high-quality results.
BibTeX
@inproceedings{zhang2024dp,
  author = {Liguo Zhang and Yalong Zhu and Achref Benarab and Yusen Ma and Yuxin Dong and Jianguo Sun},
  title = {{DP-Font: Chinese Calligraphy Font Generation Using Diffusion Model and Physical Information Neural Network}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2024},
  url = {https://openreview.net/forum?id=SkrLT0tUq6},
}
Figure