GlyphAtlas: Disentangling Stroke Detail from Layout for Scene Text Synthesis

Luozheng Qin, Changhai Zhou, Yiyi ZhouarXiv 2026 · arXiv

Generating realistic images with text—think street signs, posters, or product labels—is a notoriously difficult task for diffusion models. While recent models like FLUX have made massive strides, they still stumble when text is small, densely packed, or written in complex scripts.

A new paper from researchers at Xiamen and Fudan Universities, titled GlyphAtlas, introduces a clever architectural shift to solve this: decoupling the where (layout) from the what (stroke detail).

TL;DR

GlyphAtlas improves scene text synthesis by providing the diffusion model with a high-resolution "atlas" of character references alongside the standard layout guide. By decoupling stroke detail from the target region's size, it achieves state-of-the-art accuracy on small, dense, and complex text, even enabling zero-shot generation of languages unseen during training.

What Problem Does This Paper Address?

The current standard for "Scene Text Synthesis" involves providing a diffusion model with a glyph image—a pre-rendered version of the text placed exactly where it should appear in the final scene.

The problem is the Resolution Trap. In a glyph image, the number of pixels allocated to a character is tied to its size in the final scene. If you’re rendering a tiny "Ingredients" list on the back of a soup can, that word might only get a handful of pixels in the reference image. The model is essentially trying to reconstruct a complex character from a blurry smudge. This leads to "hallucinated" strokes, merged letters, or complete gibberish.

Methods

The authors propose GlyphAtlas, which breaks the dependency between a character's physical size in the scene and the quality of its reference.

  1. The Glyph Atlas: In addition to the standard scene-aligned glyph image (the spatial anchor), the model receives a "Glyph Atlas." This is a collection of high-resolution cells (e.g., 64x64 pixels), where each character or word is rendered clearly, regardless of how small it will appear in the final image.
  2. VAE-based Encoding: Instead of using a semantic text encoder (like CLIP or T5), which focuses on meaning, GlyphAtlas encodes these atlas cells using the diffusion model’s native VAE. Because VAEs are trained for pixel-perfect reconstruction, they preserve the fine-grained stroke details necessary for rendering.
  3. Atlas-Anchored Attention: To link the high-res atlas back to the scene, the researchers developed a custom attention mechanism. It uses an attention mask to ensure that a specific region in the image only looks at its corresponding cell in the atlas. This prevents "stroke leakage" where the model might accidentally mix up details from neighboring words.
  4. Coarse-to-Fine Conditioning: The model uses the glyph image to figure out the layout (coarse) and then "queries" the atlas to fill in the sharp stroke details (fine).

Datasets

The model was trained on a massive pool of 1.54M images, combining:

  • AnyWord-3M: A large-scale dataset for text generation.
  • Real-scene datasets: CTW, TextOCR, HierText, ReCTS, LSVT, MLT2019, and Total-Text.
  • STIPLAR: Used specifically for evaluating zero-shot multilingual performance in Japanese, Korean, and Arabic.

Key Insights and Ideas

  • Decoupling Roles: The core "aha" moment is realizing that the glyph image was being overloaded with two conflicting roles: acting as a spatial anchor and a stroke reference. By splitting these roles, the model no longer has to "guess" what a small character looks like.
  • The "Copy-Paste" Effect: Because the model learns to attend to high-res stroke references, it effectively learns a "copy-and-adapt" behavior. This is why it excels at zero-shot transfer. Even if the model never saw Arabic during training, if you give it a clear Arabic character in the atlas, it can figure out how to render those strokes into the scene.
  • Adjustable Budgets: By using "random resolution augmentation" during training, the authors made the atlas flexible. At inference time, you can lower the resolution of the atlas cells to save memory/compute without a catastrophic drop in quality.

Key Results

GlyphAtlas sets new benchmarks across the board:

  • Accuracy: On the AnyText-benchmark, it achieved a sentence accuracy (ACC) of 88.68% for English and 77.92% for Chinese, significantly outperforming the previous SOTA (FLUX-Text).
  • Small Text Mastery: On the Total-Text "small" subset (where text is <16 pixels), GlyphAtlas improved accuracy by 9.6 points over a baseline model.
  • Multilingual Zero-Shot: Despite no Arabic or Korean in its training set, GlyphAtlas outperformed specialized models on the STIPLAR benchmark by over 20 points in those languages.

Limitations and Future Work

  • Typography Consistency: The current atlas uses a uniform font. While the model is good at adapting the style of the scene, it can occasionally make errors in font weight or color because the reference atlas is "plain."
  • Inference Overhead: Appending atlas tokens to the transformer sequence increases the computational cost. While the "atlas budgeting" helps mitigate this, it’s still heavier than a glyph-only approach.
  • Future Directions: The authors suggest exploring non-uniform fonts in the atlas and further optimizing the token efficiency of the atlas-anchored attention.

For more details, check out the project page.

Summary updated

Abstract

Scene text synthesis requires stroke-accurate characters rendered in a specified layout, as well as visual coherence with the surrounding scene. To achieve this, existing methods commonly pre-render the target text at its intended locations to form a glyph image that jointly specifies layout and glyph structure. However, this representation ties the resolution of each glyph to its target region, limiting fine-grained stroke guidance for small, densely packed, or structurally complex text. To tackle this, we propose GlyphAtlas, a scene text synthesis framework that retains the glyph image as a spatial anchor and supplies fine-grained stroke references through a dedicated glyph atlas. The atlas renders and encodes each character or word in a separate cell, decoupling reference resolution from target-region size. We further introduce atlas-anchored attention, which lets each text region retrieve stroke detail only from its corresponding atlas cells to refine the coarse glyph structure provided by the glyph image. This coarse-to-fine conditioning simplifies stroke rendering while preserving explicit layout correspondence. We optimize GlyphAtlas with a mask-weighted flow matching objective and a timestep-weighted text perceptual loss for writing accuracy and background consistency. Random resolution augmentation of atlas cells supports adjustable atlas budgets at inference. Extensive experiments demonstrate that GlyphAtlas achieves stroke-accurate synthesis of small, dense, and structurally complex text, zero-shot transfer to languages unseen during task-specific training, and leading performance in scenes with many text regions. project page: https://fr0zencrane.github.io/project-glyphatlas/
BibTeX
@misc{qin2026glyphatlas,
  author = {Luozheng Qin and Changhai Zhou and Yiyi Zhou},
  title = {{GlyphAtlas: Disentangling Stroke Detail from Layout for Scene Text Synthesis}},
  year = {2026},
  eprint = {2611.20192},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.20192},
}
Figure