Abstract
Generating high-quality Chinese fonts is a labor-intensive task due to the vast number of characters and complex structural requirements. We present FontRL, a novel framework that leverages Deep Reinforcement Learning (DRL) to automatically synthesize Chinese fonts. By learning to warp standard stroke skeletons into specific artistic styles using Thin-Plate Spline (TPS) transformations, FontRL produces structurally accurate and stylistically consistent characters. Our approach demonstrates superior performance over existing image-to-image translation methods, creating results that are virtually indistinguishable from human-designed fonts.
Introduction
Creating a comprehensive Chinese font library is a monumental undertaking. While a standard English font requires only 26 letters, the basic Chinese GB2312 set contains 6,763 characters, with more extensive sets exceeding 70,000. Traditional AI approaches to font generation typically fall into two categories: image-to-image translation and sequential generation. Image-to-image methods often struggle with structural integrity, leading to blurry contours or missing components. Sequential methods, while respecting the writing process, often fail to achieve the high-fidelity visual outlines required for professional use.
The core challenge lies in maintaining topological correctness—ensuring the character is structurally "right"—while achieving stylistic consistency across thousands of complex glyphs.
The FontRL Framework
FontRL addresses these challenges by breaking the synthesis process into a "modify-and-assemble" pipeline rather than generating pixels from scratch. This approach ensures that the fundamental structure of the character is preserved by starting from a structurally perfect reference skeleton.
The framework consists of several key components:
- MPNet (Modification Parameter Estimating Network): Trained using DRL, this network acts as the system's "brain." It analyzes a "mean font" stroke and determines the optimal movement of 25 anchor points to warp the stroke into the target style.
- TPSTM (TPS Transformation Module): This module applies Thin-Plate Spline (TPS) interpolation—a non-linear warping technique—to smoothly deform the reference stroke based on the actions suggested by MPNet.
- BBoxNet (Bounding Box predicting Network): A CNN that predicts the final location and scale (bounding boxes) for each styled stroke, allowing them to be assembled into a complete character skeleton.
- IRM (Image Rendering Module): An image-to-image translation model (StyleNet) that renders the assembled skeleton into a high-resolution glyph image.
Key Insights
- Modification over Creation: Deforming a structurally sound reference is more effective than generating a character from a blank canvas. This prevents structural "breakage."
- TPS for Organic Warping: Unlike simple affine transformations, TPS allows for complex, organic warping that captures the subtle nuances of calligraphic styles.
- DRL for Parameter Optimization: Reinforcement learning is used to search for the ideal mathematical parameters of the transformation, with rewards based on the stylistic match to the target.
- Decoupling Style and Position: By processing stroke style and global positioning separately, FontRL avoids the common pitfalls of global CNNs that struggle to manage both simultaneously.
Experimental Results
The researchers evaluated FontRL using a dataset of 6,763 Chinese characters across five distinct font styles. Remarkably, the model was trained on only 775 characters (covering basic stroke types) and successfully generalized to thousands of unseen characters.
Quantitatively, FontRL achieved the lowest L1 loss and highest Intersection over Union (IoU) compared to state-of-the-art models like zi2zi, DCFont, and SCFont. Qualitatively, the generated characters exhibited sharper edges and more natural writing trajectories. In a user study with 153 participants, the accuracy in distinguishing FontRL-generated characters from human-designed ones was approximately 50.5%, indicating that the AI's output is virtually indistinguishable from professional human design.
Limitations and Future Work
While highly effective, FontRL faces challenges with highly cursive styles where strokes merge (ligatures), as it treats strokes as separate entities. Additionally, extreme stylistic differences between the reference and target fonts can lead to TPS distortion. Future work will focus on improving stroke connection handling and the robustness of the assembly phase to better capture the continuous flow of rapid handwriting.
Summary updated