From Fragment to One Piece: A Review on AI-Driven Graphic Design

Xingxing Zou, Wen Zhang, Nanxuan ZhaoJ. Imaging 2025 · DOI

TL;DR

This paper provides a comprehensive review of AI-driven graphic design (AIGD), categorizing advancements into perception and generation tasks. It highlights the shift from atomic component optimization to holistic creative systems, driven by large language models and multimodal approaches, and discusses current challenges and future directions for AI in graphic design.

What Problem Does This Paper Address?

The field of graphic design is undergoing a significant transformation with the integration of Artificial Intelligence (AI). However, existing AI approaches often focus on decomposing design tasks into isolated components (e.g., typography, layout, color) leading to fragmented outputs. The challenge is to move towards AI systems that can understand and generate complete, cohesive graphic designs, bridging the gap between human intent and the final artwork. This paper addresses the need for a unified understanding of the current state of AI in graphic design, its methodologies, challenges, and future potential.

Methods

The paper categorizes AIGD research into two primary directions:

  • Perception Tasks: These involve understanding and analyzing design elements. This includes:
  • Object Recognition: Utilizing multimodal large language models (MLLMs) and transformer-based detectors like DETR for analyzing raster and vector graphics.
  • SVG Recognition: Employing graph-matching techniques and Graph Neural Networks (GNNs) to interpret vector graphics.
  • Text Element Perception: Covering Optical Character Recognition (OCR) with techniques like TextBoxes and Mask R-CNN, and font recognition using CNNs and GANs.
  • Layout Analysis: Utilizing top-down, bottom-up, hybrid, and multi-scale approaches, often leveraging transformers.
  • Aesthetic Understanding: Focusing on color palette recommendation (using VAEAC and transformers) and assessment of other aesthetic attributes.
  • Generation Tasks: These focus on creating new design elements and layouts. This includes:
  • Non-Text Element Generation: SVG generation using seq2seq models, VAEs, and diffusion models; and vectorization of artist-generated imagery.
  • Text Element Generation: Artistic typography generation (using GANs and diffusion models) and visual text rendering (using large pre-trained models like T5).
  • Layout Generation: Automatic layout generation using templates, GANs, diffusion models, and LLMs; and glyph layout generation.
  • Colorization: Converting grayscale images to full color using various techniques including diffusion models.

The paper emphasizes the growing role of large language models (LLMs) and multimodal approaches in bridging the semantic gap between visual features and design intent.

Datasets

The paper mentions the use of various datasets implicitly through the cited research. It highlights the need for specialized datasets for graphic design tasks, noting that current datasets are often limited in scope or diversity. For instance, it mentions the creation of extensive text logo datasets in [214] and the challenges of gathering high-quality data for multilingual text rendering.

Key Insights and Ideas

  • Shift from Fragmentation to Holism: The field is moving from optimizing individual design components to creating integrated systems that maintain aesthetic consistency across entire design workflows.
  • LLMs as a Unifying Force: Large Language Models are proving crucial in understanding user intent, generating text, and even guiding layout and image generation, acting as a bridge between abstract concepts and visual outputs.
  • Vector Graphics Gaining Traction: While raster images dominate AI research, vector graphics are becoming increasingly important in AIGD due to their scalability and suitability for LLM integration.
  • Interpretability and Control Remain Key Challenges: Despite advancements, AI systems still struggle to explain their design choices and offer fine-grained control over multilayered compositions, hindering designer trust and collaboration.
  • Multimodal Integration is Crucial: Combining vision and language understanding is essential for AIGD systems to accurately interpret complex design briefs and generate relevant outputs.

Key Results

The paper doesn't present new experimental results but rather synthesizes findings from existing research. It highlights:

  • Increased Research Interest: A significant growth in AIGD research, particularly in generation tasks since 2010, with a notable surge from 2022 onwards.
  • Performance of LLMs: LLMs are showing promise in tasks like layout generation (e.g., LayoutGPT) and multimodal design tasks, though challenges in semantic understanding and visual feature loss persist.
  • Advancements in Specific Tasks: Significant progress has been made in areas like SVG generation, artistic typography, and colorization, with diffusion models and transformers playing a key role.

Limitations and Future Work

Limitations:

  • Inadequate User Intention Understanding: Current AI models often rely on statistical pattern matching rather than true semantic understanding of design requirements, leading to inconsistencies, especially in iterative feedback scenarios.
  • Limited Interpretability: AI systems struggle to articulate design rationale in terms aligned with established design principles, making it difficult for designers to understand and trust the AI's decisions.
  • Insufficient Layer Control: Generative models often lack precise control over individual elements and layer hierarchies, making complex editing and maintaining semantic consistency across layers challenging.
  • Contextual Consistency: Professional graphic design requires coherent visual systems across multiple formats. Current AI approaches often treat each generation as an independent task, failing to maintain cross-artifact consistency.
  • Data Scarcity: Particularly for multilingual tasks and specialized graphic design elements, high-quality and diverse datasets remain a challenge.

Future Work:

  • Unified End-to-End Models: Developing models that integrate multimodal intent understanding, high-quality visual element generation, and knowledge-enhanced layout reasoning.
  • Enhanced Multimodal Intent Understanding: Improving AI's ability to handle artistic images, complex layouts, specialized text effects, and ambiguous user inputs.
  • Knowledge-Enhanced Layout Reasoning: Encoding established design theories and principles into AI frameworks for more principled design decisions.
  • High-Quality Visual Element Generation: Advancing text generation capabilities, especially for artistic typography, and seamless transitions between raster and vector formats.
  • Interactive and Collaborative Design Systems: Developing systems that facilitate turn-taking, interpretable suggestions, and preserve designer agency.
  • Design Rationale Understanding: Models that can infer design intentions and explain design decisions based on underlying principles.
  • Vector Graphics Domain: Further exploration of SVG generation and reasoning capabilities.

Summary updated

BibTeX
@article{zou2025from,
  author = {Xingxing Zou and Wen Zhang and Nanxuan Zhao},
  title = {{From Fragment to One Piece: A Review on AI-Driven Graphic Design}},
  journal = {J. Imaging},
  year = {2025},
  doi = {10.3390/jimaging11090289},
}
Figure