SGCE-Font: Skeleton Guided Channel Expansion for Chinese Font Generation

Jie Zhou, Yefei Wang, Yiyang Yuan, Qing Huang, Jinshan Zeng

TL;DR

SGCE-Font introduces a "Skeleton Guided Channel Expansion" module to solve the persistent problem of mode collapse in unsupervised Chinese font generation. By integrating character skeletons directly into the generator as an additional input channel, the model provides a robust structural anchor that ensures generated characters remain legible and stylistically accurate across diverse font styles.

What Problem Does This Paper Address?

Generating Chinese fonts automatically is a massive challenge due to the sheer number of characters (often over 6,000 for standard use) and their intricate structural complexity. While Generative Adversarial Networks (GANs), particularly CycleGAN, have shown promise for unsupervised font translation, they frequently suffer from mode collapse.

In this context, mode collapse means the generator "gives up" on learning the specific structures of different characters and instead produces the same garbled or incorrect output for various inputs. Previous attempts to fix this used "stroke encodings" (which are too local and can be ambiguous) or "square-block transformations" (which are too global and coarse). As seen in the figure below, these methods often fail to capture the unique identity of a character.

Fig. 2: Examples of Chinese characters and their stroke encodings, square-block transformations and skeletons.

Methods

The authors propose the Skeleton Guided Channel Expansion (SGCE) module. The core idea is to use the character's skeleton as an explicit guide for the generator.

  1. Skeletonization: Instead of using a complex neural network, the authors use a simple, efficient mathematical thinning algorithm to extract a 1-channel skeleton from the source character image.
  2. Channel Expansion: Standard generators take a 3-channel (RGB) image. The SGCE module concatenates the 1-channel skeleton with the RGB image to create a 4-channel input. This forces the generator to consider the structural "bones" of the character from the very first layer.
  3. Skeleton Consistency Loss: To further enforce structural integrity, the authors introduce a new loss function that compares the skeleton of the generated character to the skeleton of the source character, ensuring they remain consistent.
Fig. 4: Skeleton guided channel expansion (SGCE) module for Chinese font generation.

The overall pipeline, dubbed SGCE-Font, integrates this module into a CycleGAN framework, utilizing adversarial losses, cycle consistency losses, and the new skeleton consistency loss.

Fig. 5: The pipeline of the proposed SGCE-Font model for Chinese font generation, where we take CycleGAN [23] as the base model as an example.

Datasets

The researchers evaluated their model on ten distinct Chinese font datasets:

  • Standard Printing Fonts: Heiti (ht), Fangsong (fs), Zhengkai (zk).
  • Handwriting Font: A dataset randomly collected from CASIA-HWDB1.1.
  • Pseudo-handwriting Fonts: Hupo (hp), Shuti (st), Lishu (ls).
  • Calligraphy Fonts: Huangcao (hc), Zhuxi (zx), SuShi (ss).

Notably, the Zhuxi and SuShi datasets are very small (920 and 166 samples, respectively), allowing the authors to test the model's performance in few-shot generation scenarios.

Key Insights and Ideas

  • Skeletons as "Sole Representations": Unlike stroke encodings, where two different characters might share the same encoding, a skeleton is a unique topological map of a character. It captures both local stroke details and global layout.
  • Direct vs. Implicit Guidance: Previous methods often imposed guidance on the discriminator. SGCE-Font provides direct guidance to the generator. This is more effective because the generator doesn't have to "guess" the structure based on the discriminator's feedback; the structure is provided as part of the input.
  • Plug-and-Play Efficiency: The SGCE module does not require additional learnable parameters for the skeleton extraction itself. This makes it a lightweight "plug-and-play" addition that can be added to other GAN architectures (like FUNIT or AttentionGAN) to boost their performance.

Key Results

SGCE-Font significantly outperformed baseline models across four major metrics: FID (Frechet Inception Distance), SSIM (Structural Similarity), PSNR (Peak Signal to Noise Ratio), and MSE (Mean Square Error).

  • Mode Collapse Alleviation: In tasks where standard CycleGAN completely collapsed (e.g., Hupo to Zhengkai), SGCE-Font produced highly realistic and correct characters.
  • Quantitative Gains: On average across 14 tasks, SGCE-Font achieved an FID of 30.88, compared to CycleGAN's 110.05 and StrokeGAN's 53.47.
  • Visual Quality: Qualitative results show that SGCE-Font avoids common artifacts like missing strokes or "blurry" characters that plague other models.
Fig. 1: Alleviating mode collapse by the proposed SGCE module over three generation tasks. From left to right of the tasks: Hupo font to Zhengkai font, Fangsong to Heiti, Fansgong to Lishu. The second to the last rows indicate different methods: CycleGAN [23] encounters mode collapse over these three generation tasks. CycleGAN with square-block transformations (SQ- GAN) [24] also encounters mode collapse in the task {Hupo→Zhengkai}. CycleGAN with stroke encodings (StrokeGAN) [25] generates characters with some flaws. The CycleGAN equipped with the proposed SGCE module (SGCE-Font) can effectively address the mode collapse issue and generate very realistic Chinese characters.

Limitations and Future Work

While highly effective for Chinese, the authors note that the current approach relies on a specific skeletonization strategy tailored to the strokes of Chinese characters.

Future directions include: * Adapting the SGCE module for other stroke-based languages, such as Korean (Hangul). * Further refining the model for extreme few-shot scenarios where only a handful of target font samples are available. * Exploring more sophisticated skeletonization techniques that might handle highly cursive or "messy" calligraphy even better.

Summary updated

Abstract

The automatic generation of Chinese fonts is an important problem involved in many applications. The predominated methods for the Chinese font generation are based on the deep generative models, especially the generative adversarial networks (GANs). However, existing GAN-based methods (say, CycleGAN) for the Chinese font generation usually suffer from the mode collapse issue, mainly due to the lack of effective guidance information. This paper proposes a novel information guidance module called the skeleton guided channel expansion (SGCE) module for the Chinese font generation through integrating the skeleton information into the generator with the channel expansion way, motivated by the observation that the skeleton embodies both local and global structure information of Chinese characters. We conduct extensive experiments to show the effectiveness of the proposed module. Numerical results show that the mode collapse issue suffered by the known CycleGAN can be effectively alleviated by equipping with the proposed SGCE module, and the CycleGAN equipped with SGCE outperforms the state-of-the-art models in terms of four important evaluation metrics and visualization quality. Besides CycleGAN, we also show that the suggested SGCE module can be adapted to other models for Chinese font generation as a plug-and-play module to further improve their performance.
BibTeX
@misc{zhou2022sgce,
  author = {Jie Zhou and Yefei Wang and Yiyang Yuan and Qing Huang and Jinshan Zeng},
  title = {{SGCE-Font: Skeleton Guided Channel Expansion for Chinese Font Generation}},
  year = {2022},
  eprint = {2211.14475},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2211.14475},
}

Loading related content...

Figure