Total Disentanglement of Font Images into Style and Character Class Features

Daichi Haraguchi, Wataru Shimoda, Kota Yamaguchi, Seiichi UchidaarXiv 2024 · arXiv

TL;DR

This paper introduces "total disentanglement," a novel deep learning approach that completely separates a font image into independent style and content (character class) features. By enforcing unique variance losses during training, the method provides the first empirical validation of Douglas Hofstadter’s long-standing philosophical question: "Does 'A'-ness exist?" The resulting cleanly separated features achieve state-of-the-art performance in font recognition, character recognition, and one-shot font generation.

What Problem Does This Paper Address?

In 1985, cognitive scientist and philosopher Douglas Hofstadter posed a foundational question in his book Metamagical Themas: "Do all 'A's have a common 'A'-ness?" and "Do all characters printed in a font have a common spirit (style)?" He framed this as the "vertical and horizontal problem," where characters can be arranged in a matrix grid with font styles acting as columns and character classes acting as rows.

Fig. 1. A matrix of I font styles and J characters. Our goal is to extract an (almost) identical style feature from each column i and an (almost) identical content (character class) feature from each row j.

While deep learning has made massive strides in style transfer and font generation, previous approaches typically only disentangle style while leaving content entangled, or they rely on explicit content labels (like one-hot vectors) rather than extracting a pure, continuous content feature directly from the image. Completely decomposing an arbitrary font image into a pure style vector (shared across all characters of that font) and a pure content vector (shared across all fonts for that character) without losing reconstruction fidelity has remained an elusive challenge.

Methods

The authors propose an autoencoder-based architecture consisting of an Encoder ($E$) and a Decoder ($D$). The encoder maps an input image $X_{i,j}$ into two distinct 128-dimensional feature vectors: a style feature $z^s_{i,j}$ and a content feature $z^c_{i,j}$.

Overview of the total disentanglement architecture and variance losses.

To achieve true separation, the authors implement a two-phase training strategy:

  1. Pre-training: Directly minimizing feature distances often leads to a trivial "collapsing" solution where all vectors become zero. To prevent this, the network is first trained using a standard style-transfer framework to initialize the encoder, decoder, and average feature vectors.
  2. Fine-tuning with Variance Losses: The core innovation lies in two novel loss functions introduced during fine-tuning:
  3. Style Variance Loss ($L_{style}$): Forces all style features from the same font family to converge toward their average style feature ($\bar{z}^s_i$).
  4. Content Variance Loss ($L_{content}$): Forces all content features of the same character class across different fonts to converge toward their average content feature ($\bar{z}^c_j$).

Standard Mean Absolute Error (MAE) for image reconstruction and a cross-entropy classification loss on the content feature ensure that the representations retain high structural fidelity and correct semantic class information.

Datasets

The model was trained and evaluated using three distinct datasets focusing on the 26 capital letters ('A'-'Z'):

  • Google Fonts: 2,631 fonts used for training, 315 for validation, and 295 for feature visualization.
  • Adobe Fonts: 1,132 "formal" fonts used strictly for evaluation to test generalization to unseen, standard styles.
  • Capitals64: 1,502 highly decorative and "fancy" fonts used to challenge the model's limits on irregular, creative styles (58 overlapping fonts with Google Fonts were excluded).

Key Insights and Ideas

  • Empirical Proof of "A-ness": Principal Component Analysis (PCA) visualizations reveal that style features form tight vertical lines (consistent font style across different characters) and content features form clean horizontal lines (consistent character identity across different fonts). This provides elegant empirical proof that abstract character identity ("x-ness") can be isolated from its stylistic manifestation.
PCA visualization showing the clean orthogonal separation of style and content features.
  • Platonic Shadows: The authors beautifully relate their findings to Plato's Allegory of the Cave. A content feature ("A-ness") cannot be visualized as an image on its own; it is an invisible "idea" that only becomes a visible "shadow" (the font image) when illuminated by a specific "light" (the style feature).
  • The Necessity of Explicit Variance Constraints: Without the explicit variance losses introduced during fine-tuning, the orthogonal separation of style and content completely breaks down, proving that standard reconstruction and classification objectives are insufficient for total disentanglement.

Key Results

  • Font Recognition: The model's style features achieved 77.57% accuracy on Adobe Fonts and 70.17% on Capitals64, significantly outperforming competitors like SA-VAE (57.11% / 45.81%). Crucially, font classification using the content features yielded near-random accuracy (~1.24%), proving style was successfully purged from content.
  • Character Recognition: Content features achieved 98.59% accuracy on Adobe Fonts and 87.24% on Capitals64, confirming they capture pure character identity.
  • One-Shot Font Generation: Given only a single character from a target font (e.g., just the letter 'K'), the model successfully generated the remaining 25 letters of the alphabet, outperforming or matching state-of-the-art methods across metrics like Hausdorff Distance (HD) and Intersection over Union (IoU).

Limitations and Future Work

  • Uniform Style Assumption: The model assumes a font family has a uniform style across all characters. It fails on highly inconsistent fonts where different letters utilize entirely different design motifs.
  • Highly Irregular Fonts: Performance drops on extremely intricate or distorted fonts where readability is low, occasionally causing the encoder to make content misrecognitions.
  • Blurry Artifacts: Because the model operates on pixel-based bitmap images, outputs can occasionally suffer from blurry or noisy strokes.

The authors suggest that integrating these cleanly disentangled style features into diffusion models or implicit vector representations (like SVGs) represents a promising path forward for generating sharper, more complex typography.

Summary updated

Abstract

In this paper, we demonstrate a total disentanglement of font images. Total disentanglement is a neural network-based method for decomposing each font image nonlinearly and completely into its style and content (i.e., character class) features. It uses a simple but careful training procedure to extract the common style feature from all `A'-`Z' images in the same font and the common content feature from all `A' (or another class) images in different fonts. These disentangled features guarantee the reconstruction of the original font image. Various experiments have been conducted to understand the performance of total disentanglement. First, it is demonstrated that total disentanglement is achievable with very high accuracy; this is experimental proof of the long-standing open question, ``Does `A'-ness exist?'' Hofstadter (1985). Second, it is demonstrated that the disentangled features produced by total disentanglement apply to a variety of tasks, including font recognition, character recognition, and one-shot font image generation. Code is available here: https://github.com/uchidalab/total_disentanglement
BibTeX
@misc{haraguchi2024total,
  author = {Daichi Haraguchi and Wataru Shimoda and Kota Yamaguchi and Seiichi Uchida},
  title = {{Total Disentanglement of Font Images into Style and Character Class Features}},
  year = {2024},
  eprint = {2403.12784},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2403.12784},
}
Figure