Abstract
The rapid development of generative models has significantly advanced font generation. However, limited exploration has been devoted to the evaluation and interpretability of graphical fonts. Existing quality assessment models can only provide basic visual analyses, such as recognizing clarity and brightness, without in-depth explanations. To address these limitations, we first constructed a large-scale multimodal dataset named the Diversity Font Dataset (DFD), comprising 135,000 font-text pairs. This dataset encompasses a wide range of generated font types and annotations, including language descriptions and quality assessments, thus providing a robust foundation for training and evaluating font analysis models. Based on this dataset, we developed a font agent built upon a Vision-Language Model (VLM) aiming to enhance font quality assessment and offer interpretable question-answering capabilities. Alongside the original visual encoder in VLM, we integrated an Edge-Aware Traces (EAT) module to capture detailed edge information of font strokes and components. Furthermore, we introduced a Dynamic Direct Preference Optimization (D-DPO) strategy to facilitate efficient model fine-tuning. Experimental results demonstrate that Font-Agent achieves state-of-the-art performance on the established dataset. To further evaluate the generalization ability of our algorithm, we conducted additional experiments on several public datasets. The results highlight the notable advantage of Font-Agent in both assessing the quality of generated fonts and comprehending their content.
Introduction
While generative AI can now create stunning images and custom typefaces, evaluating the quality of those fonts remains a challenge. Traditional automated tools often fail to explain why a character looks incorrect—for instance, if strokes are uneven or structural symmetry is broken. Existing Multimodal Large Language Models (MLLMs) struggle with these fine-grained details because they treat font images as natural photos, missing the subtle curvatures and stroke thicknesses that define typography.
The Diversity Font Dataset (DFD)
To address the lack of high-quality training data, we introduced the Diversity Font Dataset (DFD). This dataset consists of 135,000 font-text pairs, covering English, Chinese, and numeric characters across various styles (serif, sans-serif, handwritten, decorative). Crucially, DFD goes beyond simple binary labels, providing Mean Opinion Scores (MOS), semantic tags, and detailed language descriptions that explain font quality and specific aesthetic flaws.
Methodology
The Font-Agent framework enhances a standard Vision-Language Model (Qwen-VL) with two primary innovations designed to capture typographic nuances.
Edge-Aware Traces (EAT) Module
To help the model "see" like a typographer, the EAT module focuses on high-frequency edge information. It uses Farthest Point Sampling (FPS) to identify key points along a character's outline and applies Fast Fourier Transforms (FFT) to analyze the font in the frequency domain. This allows the model to capture minute details of strokes—such as thickness and curvature—that standard visual encoders might overlook.
Dynamic Direct Preference Optimization (D-DPO)
Fine-tuning MLLMs often leads to "hallucinations." D-DPO is a training strategy that dynamically adjusts the importance of different characters during learning. It prioritizes feedback on characters where the model is uncertain or inconsistent, ensuring the final output is both factually accurate and aesthetically grounded.
Experimental Results
Font-Agent sets a new state-of-the-art (SOTA) across multiple tasks:
- Quality Assessment: On the DFD dataset, Font-Agent achieved an average accuracy of 93.8%, significantly outperforming traditional models like Xception (87.9%) and general-purpose MLLMs like GPT-4o (62.5%).
- Font Recognition: In recognizing text within complex or irregular images, Font-Agent achieved an average accuracy of 91.32% across various benchmarks (e.g., IIIT5k, Total Text).
- Interpretability: The model provides coherent, expert-style feedback, identifying specific issues like uneven leg lengths or visual imbalances.
Conclusion
Font-Agent demonstrates that explicitly processing high-frequency components and utilizing self-consistency as a training signal can significantly enhance an AI's typographic understanding. Future work will focus on expanding dataset coverage to more diverse languages and refining feedback to suggest specific geometric corrections for font designers.