FET-GAN: Font and Effect Transfer via K-shot Adaptive Instance Normalization

Wei Li, Yongxing He, Yanwei Qi, Zejian Li, Yongchuan Tang

Introduction

In graphic design, applying consistent visual effects—colors, outlines, textures, and specific font weights—across a large set of characters is a grueling, manual task. This is especially true for languages with massive character sets, like Chinese or Japanese. Existing "text effect transfer" methods often struggle with font inconsistency and poor generalization to unseen styles.

FET-GAN (Font and Effect Transfer GAN) addresses these challenges by treating typeface as a style attribute rather than a structural one. This unified framework handles both font and artistic effects simultaneously, requiring only a "few-shot" (K-shot) set of examples to learn a new style.

Methodology

The FET-GAN architecture relies on a decomposition of features, assuming a visual object consists of a global structure (identity) and local effects (typeface and textures).

The FET-GAN framework consists of an encoder that extracts an effect code and a generator that applies it via Adaptive Instance Normalization (AdaIN).

Key components include:

  • Encoder-Generator with AdaIN: An encoder ($E$) extracts an "effect code" from $K$ reference samples. This code is injected into a Generator ($G$) via Adaptive Instance Normalization (AdaIN), allowing the generator to "paint" the style onto a source character's structure.
  • Multi-class Patch Discriminator: This discriminator uses multiple channels—one for each style domain—to help the model learn nuances across many different styles simultaneously.
  • Few-shot Finetuning: For brand-new styles, the model is briefly finetuned. Parameters are initialized using the most similar existing style in the discriminator to accelerate convergence.

Results and Discussion

FET-GAN demonstrates significant visual superiority over existing methods like CycleGAN and StarGAN. It is uniquely capable of accurately transferring both complex textures and specific typefaces.

FET-GAN demonstrates the ability to transfer complex artistic effects and font styles to both text and non-text graphic objects like logos.

Key findings include:

  • Typeface as Style: By treating typeface as a "local effect," the model avoids the need for separate font sub-networks.
  • K-shot Stability: Using multiple reference images ($K > 1$) significantly stabilizes the effect code, with 4-shot or 8-shot references providing the most consistent results.
  • Cross-Domain Versatility: The model can transfer text effects onto non-text objects, such as icons, while maintaining their original shapes.
  • Font Library Completion: FET-GAN successfully generated missing characters for a font library after seeing only four example characters, highlighting its potential for reducing manual labor in font design.

Conclusion

While FET-GAN offers a powerful solution for style transfer, it faces challenges such as potential overfitting during finetuning if iterations are not carefully managed. Future work may involve automating the selection of finetuning iterations and managing the architectural complexity as the number of supported styles grows.

Summary updated

Abstract

Text effect transfer aims at learning the mapping between text visual effects while maintaining the text content. While remarkably successful, existing methods have limited robustness in font transfer and weak generalization ability to unseen effects. To address these problems, we propose FET-GAN, a novel end-to-end framework to implement visual effects transfer with font variation among multiple text effects domains. Our model achieves remarkable results both on arbitrary effect transfer between texts and effect translation from text to graphic objects. By a few-shot fine-tuning strategy, FET-GAN can generalize the transfer of the pre-trained model to the new effect. Through extensive experimental validation and comparison, our model advances the state-of-the-art in the text effect transfer task. Besides, we have collected a font dataset including 100 fonts of more than 800 Chinese and English characters. Based on this dataset, we demonstrated the generalization ability of our model by the application that complements the font library automatically by few-shot samples. This application is significant in reducing the labor cost for the font designer.
BibTeX
@inproceedings{li2020fet,
  author = {Wei Li and Yongxing He and Yanwei Qi and Zejian Li and Yongchuan Tang},
  title = {{FET-GAN: Font and Effect Transfer via K-shot Adaptive Instance Normalization}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2020},
  doi = {10.1609/aaai.v34i02.5535},
}
Figure