Abstract
Visual layouts are foundational to graphic design, yet traditional generative models often fail to capture the contextual aesthetic requirements necessary for high-quality output. These models typically treat layout generation as a coordinate prediction task, ignoring the visual harmony of the final rendered result. To address this, we introduce AesthetiQ, a framework utilizing Aesthetic-Aware Preference Alignment (AAPA). By leveraging Multi-modal Large Language Models (MLLMs) and Direct Preference Optimization (DPO), AesthetiQ aligns layout generation with human-like aesthetic preferences. Our approach includes a data filtering protocol based on layout-quality heuristics and a novel evaluation metric that uses a "Judge" MLLM to determine win rates against ground-truth designs. Experimental results on the Crello and WebUI benchmarks demonstrate that AesthetiQ outperforms current state-of-the-art (SOTA) models by up to 17%, highlighting the potential of MLLMs in creating visually coherent and aesthetically pleasing designs.
Introduction
Automated layout generation—the strategic placement of text, images, and shapes—is a cornerstone of modern digital design. Historically, this has been approached as a numerical optimization problem where models are trained to minimize the distance between predicted coordinates $(x, y, w, h)$ and a ground-truth template.
However, design is inherently subjective. A model might produce a layout that is geometrically different from the training data but aesthetically superior, yet traditional cross-entropy loss functions would penalize such variations. Conversely, models often produce "accurate" coordinates that result in visual clutter, overlapping elements, or poor hierarchy because they do not "see" the final rendered output. AesthetiQ shifts this paradigm by imbuing MLLMs with a sense of visual "vibe" and harmony through preference-based learning.
The AesthetiQ Framework
AesthetiQ utilizes the InternVL architecture and follows a three-stage pipeline designed to move beyond simple coordinate matching toward aesthetic intelligence.
1. Position-Aware Instruction Tuning
The model is first trained to interpret spatial relationships by discretizing continuous coordinates into discrete "position tokens." This transformation allows the MLLM to process layout generation as a sequence-to-sequence language task, leveraging the pre-existing reasoning capabilities of large-scale language models.
2. Aesthetic-Aware Preference Alignment (AAPA)
This core innovation employs Direct Preference Optimization (DPO) to refine the model’s output. Rather than simply imitating a single ground truth, the model is presented with pairs of generated layouts. A separate, high-capacity "Judge" MLLM (such as VILA-7B) evaluates these pairs based on professional design criteria: clarity, usability, visual consistency, and hierarchy. By learning to prefer the more aesthetically pleasing option, AesthetiQ aligns its generative process with human-centric design principles.
3. Quality-Metrics Data Filtering
To prevent the model from learning from low-quality examples, the researchers implemented a rigorous filtering protocol. Using heuristics to measure element alignment and minimize unintended overlaps, the framework ensures that only high-quality, well-structured layouts are utilized during the preference learning phase.
Evaluation and Benchmarks
The efficacy of AesthetiQ was tested on two diverse datasets:
- Crello: 23,302 professional design templates for social media and advertising.
- WebUI: 70,000 web page interfaces featuring complex, hierarchical structures.
In addition to traditional metrics like Mean Intersection over Union (mIoU), the authors introduced the MLLM Win Rate. In this setup, a powerful vision model (e.g., GPT-4o) acts as a critic to determine how often the AesthetiQ-generated layout is preferred over the original human-designed ground truth.
Key Insights and Results
Superior Performance
AesthetiQ achieved a new state-of-the-art across both benchmarks. On the Crello dataset, AesthetiQ-8B reached a 42.83% Mean IoU, a significant improvement over previous leaders like LayoutNUWA (25.74%) and LACE (23.18%). On the WebUI dataset, the model achieved a 24.48% Win Rate against ground truth, nearly doubling the performance of existing multi-modal approaches.
The Value of Visual Feedback
The research demonstrates that MLLMs can effectively serve as aesthetic critics. Traditional metrics like IoU are often "blind" to design harmony; AesthetiQ proves that a model can achieve a lower IoU (indicating it did not copy the original exactly) while achieving a higher Win Rate because its generated layout is more modern or visually balanced than the original template.
Scaling and Aesthetics
The study found a clear correlation between model scale and design quality. The 8B parameter version of AesthetiQ demonstrated a more sophisticated grasp of subtle spatial nuances and complex element relationships compared to the 1B parameter variant, suggesting that larger models are better equipped to handle the nuances of visual aesthetics.
Conclusion and Future Work
AesthetiQ represents a significant leap in content-aware layout generation by integrating visual preference into the training loop. While the model excels at spatial arrangement, future iterations could expand this framework to include other design dimensions such as font selection and color palette coordination. By moving from strict coordinate regression to aesthetic alignment, AesthetiQ paves the way for AI systems that truly understand the "art" of design.
Summary updated