Abstract
Few-shot font generation (FFG) for glyph-rich scripts like Chinese is a labor-intensive task that current generative models struggle to automate without structural artifacts. We propose MSD-Font, a framework that rethinks FFG by mimicking the systematic workflow of human designers. By decomposing the generation process into three distinct stages—Structure Construction, Font Transfer, and Font Refinement—within a Latent Diffusion Model (LDM) framework, we achieve high-fidelity font synthesis. Our approach utilizes a dual-network architecture to specialize in style adaptation and detail polishing, significantly outperforming existing GAN-based state-of-the-art methods in both quantitative metrics and visual quality.
Introduction
Few-shot font generation (FFG) aims to create a complete stylized font set from a limited number of reference images. This is particularly critical for scripts like Chinese, which contain thousands of unique characters. Traditional methods primarily rely on Generative Adversarial Networks (GANs) using a style-content disentanglement paradigm. However, these "all-at-once" approaches often fail to balance complex global structures with intricate local styles, leading to broken characters or inconsistent aesthetics. MSD-Font addresses these limitations by adopting a multi-stage diffusion process that mirrors the step-by-step refinement used by professional designers.
Methodology
MSD-Font is built upon Latent Diffusion Models (LDM), where the reverse diffusion process is strategically decomposed into three stages:
- Structure Construction Stage: The model generates the global structure of the target character using a source template as an anchor. This ensures topological correctness from the start.
- Font Transfer Stage: Within the time interval $(t_1, t_2]$, the model transforms the latent features of the source font into the target style. This is treated as a domain mapping task.
- Font Refinement Stage: In the final steps $(0, t_1]$, the model focuses on recovering fine-grained local details and polishing the final appearance.
To effectively manage these distinct tasks, the authors implement a dual-network approach. A single network often fails to master both style transfer and detail refinement simultaneously. By employing two specialized prediction networks ($z_{ heta1}$ and $z_{ heta2}$), the model achieves higher precision in both global style adaptation and local stroke recovery.
Experiments and Results
The model was evaluated using a dataset of 900 Chinese fonts. Training was conducted on 840 fonts, while testing focused on two challenging scenarios: Unseen Fonts, Unseen Contents (UFUC) and Unseen Fonts, Seen Contents (UFSC).
MSD-Font outperformed prominent state-of-the-art (SOTA) methods, including LF-Font, MX-Font, and FsFont. On the UFUC dataset, MSD-Font achieved a structural similarity (SSIM) of 0.7189 and a perceptual similarity (LPIPS) of 0.1527, surpassing its closest competitors.
Qualitative analysis reveals that MSD-Font produces significantly cleaner strokes and more stable structures. While GAN-based methods frequently produce blurry or "broken" characters for complex glyphs, the multi-stage diffusion approach maintains character integrity while accurately applying the target style.
Conclusion
MSD-Font demonstrates that incorporating a structured, multi-stage process into diffusion models can solve the long-standing issues of structural instability in few-shot font generation. While the iterative nature of diffusion models introduces higher computational costs compared to GANs, the gains in visual fidelity and stylistic consistency are substantial. Future work will focus on optimizing inference speed and exploring cross-lingual applications.
Summary updated