Abstract: The automatic generation of Chinese fonts is an important problem involved in many applications. The predominated methods for the Chinese font generation are based on the deep generative models, especially the generative adversarial networks (GANs). However, existing GAN-based methods (say, CycleGAN) for the Chinese font generation usually suffer from the mode collapse issue, mainly due to the lack of effective guidance information. This paper proposes a novel information guidance module called the skeleton guided channel expansion (SGCE) module for the Chinese font generation through integrating the skeleton information into the generator with the channel expansion way, motivated by the observation that the skeleton embodies both local and global structure information of Chinese characters. We conduct extensive experiments to show the effectiveness of the proposed module. Numerical results show that the mode collapse issue suffered by the known CycleGAN can be effectively alleviated by equipping with the proposed SGCE module, and the CycleGAN equipped with SGCE outperforms the state-of-the-art models in terms of four important evaluation metrics and visualization quality. Besides CycleGAN, we also show that the suggested SGCE module can be adapted to other models for Chinese font generation as a plug-and-play module to further improve their performance.
Addressing the Mode Collapse in Chinese Font Generation
Generating Chinese fonts is a massive undertaking because the language contains tens of thousands of unique characters. While deep learning models like CycleGAN can learn to translate styles between fonts without needing perfectly paired datasets (unsupervised learning), they often stumble over the structural complexity of Chinese characters.
The most common failure is mode collapse, where the generator "gives up" on learning the specific structure of a character and instead outputs a generic or incorrect shape that happens to fool the discriminator. Previous attempts to fix this used "stroke encodings" (digital IDs for strokes) or "square-block transformations," but these often aren't detailed enough to distinguish between similar-looking characters.
The Skeleton Guided Channel Expansion (SGCE) Module
The authors propose the Skeleton Guided Channel Expansion (SGCE) module as an elegantly simple solution to provide explicit structural guidance to the generator. The process involves three main components:
- Skeletonization: Using a classic mathematical thinning algorithm (Zhang-Suen), the model extracts a one-pixel-wide "skeleton" from the source character. This skeleton captures both the global layout and local stroke details.
- Channel Expansion: Instead of feeding the generator a standard 3-channel RGB image, the SGCE module appends the skeleton as a 4th channel. This provides the generator with an explicit structural roadmap right from the start.
- Skeleton Consistency Loss: To ensure the model doesn't "drift" during training, a specific loss function penalizes the model if the skeleton of the generated character doesn't match the skeleton of the input.
Because it only modifies the input layer and adds a loss term, the SGCE module is a "plug-and-play" architecture that can be bolted onto almost any existing GAN architecture (like CycleGAN, UGATIT, or FUNIT) to improve its performance.
Experimental Evaluation
The researchers evaluated their model across 10 different font datasets, covering standard printing (Heiti, Fangsong), handwriting (CASIA-HWDB1.1), and artistic styles (Hupo, Shuti). A key insight of this work is that giving the generator structural info directly via the 4th channel is far more effective at preventing errors than giving the discriminator extra info, as many previous models attempted.
Key Results
The SGCE-Font model was tested against six state-of-the-art baselines, yielding striking results:
- Eliminating Mode Collapse: In tasks where standard CycleGAN and other baselines failed completely (producing garbled characters), the SGCE-equipped version produced near-perfect characters.
- Quantitative Superiority: On the FID metric (measuring realism), SGCE-Font improved CycleGAN’s average score from 110.05 down to 30.88.
- Generalization: When added to other models like AttentionGAN or FUNIT, the SGCE module consistently improved Structural Similarity (SSIM) and Peak Signal-to-Noise Ratio (PSNR) scores.
Conclusion and Future Work
By treating skeletons as "structural DNA," SGCE-Font provides a robust anchor that prevents models from collapsing into incorrect modes. The skeleton extraction is a non-neural, mathematical process, meaning the model gains significant structural intelligence without a massive increase in computational overhead.
While highly effective, future work may involve expanding this approach to other character-based languages like Korean (Hangul) and further optimizing the module for "extreme few-shot" scenarios, where a whole font must be generated from just a handful of examples.
Summary updated