AesthetiQ: Enhancing Graphic Layout Design via Aesthetic-Aware Preference Alignment of Multi-modal Large Language Models

Sohan Patnaik, Rishabh Jain, Balaji Krishnamurthy, Mausoom Sarkar

Abstract

Visual layouts are foundational to graphic design, yet traditional generative models often fail to capture the contextual aesthetic requirements necessary for high-quality output. These models typically treat layout generation as a coordinate prediction task, ignoring the visual harmony of the final rendered result. To address this, we introduce AesthetiQ, a framework utilizing Aesthetic-Aware Preference Alignment (AAPA). By leveraging Multi-modal Large Language Models (MLLMs) and Direct Preference Optimization (DPO), AesthetiQ aligns layout generation with human-like aesthetic preferences. Our approach includes a data filtering protocol based on layout-quality heuristics and a novel evaluation metric that uses a "Judge" MLLM to determine win rates against ground-truth designs. Experimental results on the Crello and WebUI benchmarks demonstrate that AesthetiQ outperforms current state-of-the-art (SOTA) models by up to 17%, highlighting the potential of MLLMs in creating visually coherent and aesthetically pleasing designs.

Introduction

Automated layout generation—the strategic placement of text, images, and shapes—is a cornerstone of modern digital design. Historically, this has been approached as a numerical optimization problem where models are trained to minimize the distance between predicted coordinates $(x, y, w, h)$ and a ground-truth template.

However, design is inherently subjective. A model might produce a layout that is geometrically different from the training data but aesthetically superior, yet traditional cross-entropy loss functions would penalize such variations. Conversely, models often produce "accurate" coordinates that result in visual clutter, overlapping elements, or poor hierarchy because they do not "see" the final rendered output. AesthetiQ shifts this paradigm by imbuing MLLMs with a sense of visual "vibe" and harmony through preference-based learning.

The AesthetiQ Framework

AesthetiQ utilizes the InternVL architecture and follows a three-stage pipeline designed to move beyond simple coordinate matching toward aesthetic intelligence.

1. Position-Aware Instruction Tuning

The model is first trained to interpret spatial relationships by discretizing continuous coordinates into discrete "position tokens." This transformation allows the MLLM to process layout generation as a sequence-to-sequence language task, leveraging the pre-existing reasoning capabilities of large-scale language models.

2. Aesthetic-Aware Preference Alignment (AAPA)

This core innovation employs Direct Preference Optimization (DPO) to refine the model’s output. Rather than simply imitating a single ground truth, the model is presented with pairs of generated layouts. A separate, high-capacity "Judge" MLLM (such as VILA-7B) evaluates these pairs based on professional design criteria: clarity, usability, visual consistency, and hierarchy. By learning to prefer the more aesthetically pleasing option, AesthetiQ aligns its generative process with human-centric design principles.

3. Quality-Metrics Data Filtering

To prevent the model from learning from low-quality examples, the researchers implemented a rigorous filtering protocol. Using heuristics to measure element alignment and minimize unintended overlaps, the framework ensures that only high-quality, well-structured layouts are utilized during the preference learning phase.

Evaluation and Benchmarks

The efficacy of AesthetiQ was tested on two diverse datasets:

  • Crello: 23,302 professional design templates for social media and advertising.
  • WebUI: 70,000 web page interfaces featuring complex, hierarchical structures.

In addition to traditional metrics like Mean Intersection over Union (mIoU), the authors introduced the MLLM Win Rate. In this setup, a powerful vision model (e.g., GPT-4o) acts as a critic to determine how often the AesthetiQ-generated layout is preferred over the original human-designed ground truth.

Key Insights and Results

Superior Performance

AesthetiQ achieved a new state-of-the-art across both benchmarks. On the Crello dataset, AesthetiQ-8B reached a 42.83% Mean IoU, a significant improvement over previous leaders like LayoutNUWA (25.74%) and LACE (23.18%). On the WebUI dataset, the model achieved a 24.48% Win Rate against ground truth, nearly doubling the performance of existing multi-modal approaches.

The Value of Visual Feedback

The research demonstrates that MLLMs can effectively serve as aesthetic critics. Traditional metrics like IoU are often "blind" to design harmony; AesthetiQ proves that a model can achieve a lower IoU (indicating it did not copy the original exactly) while achieving a higher Win Rate because its generated layout is more modern or visually balanced than the original template.

Scaling and Aesthetics

The study found a clear correlation between model scale and design quality. The 8B parameter version of AesthetiQ demonstrated a more sophisticated grasp of subtle spatial nuances and complex element relationships compared to the 1B parameter variant, suggesting that larger models are better equipped to handle the nuances of visual aesthetics.

Conclusion and Future Work

AesthetiQ represents a significant leap in content-aware layout generation by integrating visual preference into the training loop. While the model excels at spatial arrangement, future iterations could expand this framework to include other design dimensions such as font selection and color palette coordination. By moving from strict coordinate regression to aesthetic alignment, AesthetiQ paves the way for AI systems that truly understand the "art" of design.

Summary updated

Abstract

Visual layouts are essential in graphic design fields such as advertising, posters, and web interfaces. The application of generative models for content-aware layout generation has recently gained traction. However, these models fail to understand the contextual aesthetic requirements of layout design and do not align with human-like preferences, primarily treating it as a prediction task without considering the final rendered output. To overcome these problems, we offer Aesthetic-Aware Preference Alignment(AAPA), a novel technique to train a Multi-modal Large Language Model (MLLM) for layout prediction that uses MLLM's aesthetic preferences for Direct Preference Optimization over graphic layouts. We propose a data filtering protocol utilizing our layout-quality heuristics for AAPA to ensure training happens on high-quality layouts. Additionally, we introduce a novel evaluation metric that uses another MLLM to compute the win rate of the generated layout against the ground-truth layout based on aesthetics criteria. We also demonstrate the applicability of AAPA for MLLMs of varying scales (1B to 8B parameters) and LLM families (Qwen, Phi, InternLM). By conducting thorough qualitative and quantitative analyses, we verify the efficacy of our approach on two challenging benchmarks - Crello and Webui, showcasing 17%, and 16 improvement over current State-of-The-Art methods, thereby highlighting the potential of MLLMs in aesthetic-aware layout generation.
BibTeX
@inproceedings{patnaik2025aesthetiq,
  author = {Sohan Patnaik and Rishabh Jain and Balaji Krishnamurthy and Mausoom Sarkar},
  title = {{AesthetiQ: Enhancing Graphic Layout Design via Aesthetic-Aware Preference Alignment of Multi-modal Large Language Models}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2025},
  url = {https://openreview.net/forum?id=IUWWviXQCC},
}
Figure