TextMaster: A Unified Framework for Realistic Text Editing via Glyph-Style Dual-Control

Zhenyu Yan, Jian Wang, Aoqiang Wang, Yuhan Li, Wenxiang Shang, Zhu HangchengICCV 2025

TL;DR

This paper introduces TextMaster, a novel framework for realistic text editing in images that precisely controls both the visual appearance of characters (glyphs) and their stylistic attributes. By combining character-level constraints, adaptive layout mechanisms, and a style injection technique, TextMaster achieves state-of-the-art performance in generating accurate, stylistically consistent, and well-placed text edits across diverse scenarios.

What Problem Does This Paper Address?

Editing text within images is a challenging task that requires more than just generating plausible text. Existing methods often struggle with:

  • Stroke Accuracy for Complex Text: Generating intricate character strokes, especially in non-Latin scripts, with high fidelity is difficult.
  • Controllability of Text Styles: Achieving precise control over the visual style of the edited text, such as font, color, gradients, and outlines, while maintaining consistency with the surrounding image context is a significant hurdle.
  • Layout and Alignment: Ensuring the generated text is correctly positioned and aligned within the image, especially in complex layouts or when editing small text regions.
  • Authenticity and Diversity: Many existing methods rely on synthesized datasets or struggle to handle the natural diversity and authenticity of real-world text images.

These limitations hinder the practical application of text editing in areas like graphic design, content creation, and image manipulation.

Methods

TextMaster tackles these challenges through a Glyph-Style Dual-Control framework, integrating several key components:

  • Glyph Control:
    • Glyph Constraint: Leverages standard character images and precise textual encoding. It uses a quadrilateral mask as a foundation for glyph representation and incorporates perceptual loss to ensure accurate stroke formation.
  • Position-aware Attention: This mechanism helps the model learn character positioning by analyzing attention maps from specific cross-attention layers and calculating IoU overlap with ground-truth bounding boxes. This optimizes layout learning.
  • Token-separated Text Encoder: A modified text encoder (ChatGLM) is used to handle semantic relationships in text, preventing context from adversely affecting character embeddings and maintaining uniform character representations.
  • Perceptual Loss: A combination of OCR feature L2 loss and MSE loss is used to enforce consistency in glyph structures and ensure natural transitions between text and background.
  • Style Control:
    • Style-Adapter: This module decouples font style from glyph content. It extracts style information by subtracting content features from style features of cropped text regions. This extracted style is then injected into the UNet's cross-attention layers.
  • In-Context Reference: To address chromatic deviations and improve style understanding, this technique uses self-attention mechanisms to learn spatial-semantic relationships between masked regions and a contextual reference text. This helps the model capture detailed stylistic features.

The overall framework integrates these modules with a diffusion model (Stable Diffusion) for text editing.

Datasets

  • Training: The model was trained on the AnyWord-3M dataset, which combines subsets from LAION and Wukong. Additionally, 100,000 artistic style posters were crawled from the web to enhance the model's style infusion capabilities.
  • Evaluation: The model was evaluated on the ICDAR13 dataset, the Anytext benchmark, and a newly built textmaster dataset.

Key Insights and Ideas

  • Decoupling Glyph and Style is Crucial: The paper highlights that effectively separating the structural integrity of characters (glyphs) from their visual appearance (style) is key to achieving both accuracy and stylistic control.
  • Character-Level Control for Layout: Using position-aware attention and IoU loss for character bounding boxes significantly improves the model's ability to generate text with proper typographic arrangement and avoid layout issues.
  • Perceptual Loss for Fidelity: Incorporating perceptual loss, specifically using OCR features, acts as a strong supervisory signal to ensure the reconstructed text region accurately reflects the original glyph structures.
  • Style Injection for Authenticity: The Style-Adapter and In-Context Reference mechanisms enable the model to learn and inject nuanced stylistic features from reference images, leading to more visually harmonious and contextually appropriate text edits.
  • Handling Small Text Regions: The Adaptive Crop and Mask Strategy is crucial for maintaining text quality and background restoration, especially in very small text regions where standard VAE encoding might degrade glyph structures.

Key Results

  • State-of-the-Art Performance: TextMaster outperforms existing methods across multiple benchmarks and metrics, including Accuracy (ACC), NED, FID, and LPIPS, for both Chinese and English text editing.
  • Quantitative Improvements: Table 1 shows TextMaster achieving high accuracy (e.g., 0.9257 ACC in Chinese) and low FID/LPIPS scores (e.g., 11.93 FID in Chinese), indicating superior content accuracy and style preservation.
  • Qualitative Superiority: Figure 6 and other qualitative results demonstrate TextMaster's ability to handle complex text editing tasks, maintain style consistency, and produce visually appealing results, especially in challenging scenarios.
  • Ablation Study Success: Table 2 shows that each component of TextMaster (glyph information, perceptual losses, token-separated encoder, attention IoU loss, style adapter) contributes significantly to the overall performance.

Limitations and Future Work

  • Limitations:
    • The paper notes that while the model handles complex styles, chromatic deviations can still occur due to the model's tendency to derive color information from surrounding contexts.
  • The reliance on specific text encoders (like ChatGLM) might introduce language-specific considerations.
  • Future Work:
    • The authors suggest that the framework is not limited to specific languages and can be extended to others.
  • Further exploration of style injection and its impact on various text types and image backgrounds is implied.
  • The adaptive mask augmentation technique aims to improve robustness to bbox variations, suggesting ongoing work in refining layout control.

Summary updated

BibTeX
@inproceedings{yan2025textmaster,
  author = {Zhenyu Yan and Jian Wang and Aoqiang Wang and Yuhan Li and Wenxiang Shang and Zhu Hangcheng},
  title = {{TextMaster: A Unified Framework for Realistic Text Editing via Glyph-Style Dual-Control}},
  booktitle = {IEEE/CVF International Conference on Computer Vision},
  year = {2025},
}
Figure