TL;DR
SGCE-Font introduces a "Skeleton Guided Channel Expansion" module to solve the persistent problem of mode collapse in unsupervised Chinese font generation. By integrating character skeletons directly into the generator as an additional input channel, the model provides a robust structural anchor that ensures generated characters remain legible and stylistically accurate across diverse font styles.
What Problem Does This Paper Address?
Generating Chinese fonts automatically is a massive challenge due to the sheer number of characters (often over 6,000 for standard use) and their intricate structural complexity. While Generative Adversarial Networks (GANs), particularly CycleGAN, have shown promise for unsupervised font translation, they frequently suffer from mode collapse.
In this context, mode collapse means the generator "gives up" on learning the specific structures of different characters and instead produces the same garbled or incorrect output for various inputs. Previous attempts to fix this used "stroke encodings" (which are too local and can be ambiguous) or "square-block transformations" (which are too global and coarse). As seen in the figure below, these methods often fail to capture the unique identity of a character.
Methods
The authors propose the Skeleton Guided Channel Expansion (SGCE) module. The core idea is to use the character's skeleton as an explicit guide for the generator.
- Skeletonization: Instead of using a complex neural network, the authors use a simple, efficient mathematical thinning algorithm to extract a 1-channel skeleton from the source character image.
- Channel Expansion: Standard generators take a 3-channel (RGB) image. The SGCE module concatenates the 1-channel skeleton with the RGB image to create a 4-channel input. This forces the generator to consider the structural "bones" of the character from the very first layer.
- Skeleton Consistency Loss: To further enforce structural integrity, the authors introduce a new loss function that compares the skeleton of the generated character to the skeleton of the source character, ensuring they remain consistent.
The overall pipeline, dubbed SGCE-Font, integrates this module into a CycleGAN framework, utilizing adversarial losses, cycle consistency losses, and the new skeleton consistency loss.
Datasets
The researchers evaluated their model on ten distinct Chinese font datasets:
- Standard Printing Fonts: Heiti (ht), Fangsong (fs), Zhengkai (zk).
- Handwriting Font: A dataset randomly collected from CASIA-HWDB1.1.
- Pseudo-handwriting Fonts: Hupo (hp), Shuti (st), Lishu (ls).
- Calligraphy Fonts: Huangcao (hc), Zhuxi (zx), SuShi (ss).
Notably, the Zhuxi and SuShi datasets are very small (920 and 166 samples, respectively), allowing the authors to test the model's performance in few-shot generation scenarios.
Key Insights and Ideas
- Skeletons as "Sole Representations": Unlike stroke encodings, where two different characters might share the same encoding, a skeleton is a unique topological map of a character. It captures both local stroke details and global layout.
- Direct vs. Implicit Guidance: Previous methods often imposed guidance on the discriminator. SGCE-Font provides direct guidance to the generator. This is more effective because the generator doesn't have to "guess" the structure based on the discriminator's feedback; the structure is provided as part of the input.
- Plug-and-Play Efficiency: The SGCE module does not require additional learnable parameters for the skeleton extraction itself. This makes it a lightweight "plug-and-play" addition that can be added to other GAN architectures (like FUNIT or AttentionGAN) to boost their performance.
Key Results
SGCE-Font significantly outperformed baseline models across four major metrics: FID (Frechet Inception Distance), SSIM (Structural Similarity), PSNR (Peak Signal to Noise Ratio), and MSE (Mean Square Error).
- Mode Collapse Alleviation: In tasks where standard CycleGAN completely collapsed (e.g., Hupo to Zhengkai), SGCE-Font produced highly realistic and correct characters.
- Quantitative Gains: On average across 14 tasks, SGCE-Font achieved an FID of 30.88, compared to CycleGAN's 110.05 and StrokeGAN's 53.47.
- Visual Quality: Qualitative results show that SGCE-Font avoids common artifacts like missing strokes or "blurry" characters that plague other models.
Limitations and Future Work
While highly effective for Chinese, the authors note that the current approach relies on a specific skeletonization strategy tailored to the strokes of Chinese characters.
Future directions include: * Adapting the SGCE module for other stroke-based languages, such as Korean (Hangul). * Further refining the model for extreme few-shot scenarios where only a handful of target font samples are available. * Exploring more sophisticated skeletonization techniques that might handle highly cursive or "messy" calligraphy even better.
Summary updated