Advancing vision-language models in front-end development via data synthesis

Tong Ge, Yashu Liu, Jieping Ye, Tianyi Li, Chao WangarXiv 2025 · DOI · arXiv

Abstract

Modern front-end development requires modularity, state management, and declarative rendering—complexities where general-purpose vision-language models (VLMs) often fall short. We introduce Flame, a specialized VLM optimized for generating professional-grade React code from design mockups. Flame is trained on over 400,000 high-quality, synthetic code-image pairs generated through a novel reflective agentic workflow. Our results demonstrate that Flame significantly outperforms frontier models like GPT-4o and Gemini 1.5 Flash on the Flame-React-Eval benchmark, achieving up to a 10x improvement in functional correctness and industry-standard modularity.

Introduction

While general VLMs can generate basic HTML/CSS, they often fail at the nuances of modern front-end (FE) development. Real-world FE work involves breaking UIs into reusable components, managing dynamic states, and ensuring declarative rendering logic. Existing models frequently produce "monolithic" code—static blocks that are difficult to maintain and ignore component-based architectures. The primary bottleneck is the scarcity of high-quality, paired data linking complex UI designs with professional, self-contained code. Flame addresses this by utilizing a self-reflective agentic workflow to synthesize high-fidelity training data.

Methods and Data Synthesis

The core of Flame's success lies in its self-reflective agentic workflow, which automates the extraction and refinement of code snippets from GitHub. To expand this dataset, the authors employed three distinct synthesis strategies designed to mimic real-world development patterns:

  • Evolution-based Synthesis: Evolving existing code by adding features or increasing technical complexity (e.g., upgrading state management logic).
  • Waterfall-Model-based Synthesis: Following a traditional lifecycle from high-level requirements to UI design, architecture, and finally, logically coherent code.
  • Additive Development Synthesis: Iteratively adding complexity to simple components, such as introducing search bars or dark mode functionality.
The Flame data synthesis pipeline, illustrating the Evolution-based, Waterfall-Model-based, and Additive Development synthesis strategies.

Flame's architecture utilizes a SigLIP vision encoder and a DeepSeek-coder language base, connected via a 2-layer MLP. Training proceeds in three stages: connector warm-up, layout description learning, and final code generation fine-tuning.

Key Insights

A critical finding in this research is the value of "Interpretation Before Coding." Forcing the model to describe the visual layout before generating code acts as a visual Chain-of-Thought, significantly improving the model's understanding of visual hierarchies. Furthermore, the study found that structured synthesis (Waterfall and Additive methods) outperformed random evolution, suggesting that logical progression is more valuable for training than simple variation.

Results

Flame was evaluated on Flame-React-Eval, a benchmark of 80 manually curated test cases assessing syntactic precision and visual consistency. Flame variants achieved state-of-the-art results, far surpassing much larger proprietary models.

Performance comparison of Flame variants against state-of-the-art general-purpose models on the Flame-React-Eval benchmark.

As shown in the results, the Flame-Waterfall-7B model achieved a 52.6% pass@1 rate, while Flame-Additive-7B reached a 71.9% pass@5 rate. In comparison, GPT-4o and Gemini 1.5 Flash struggled significantly, with pass@1 rates of 4.9% and 11.1%, respectively. This highlights that a specialized 7B model trained on high-quality synthetic data can outperform frontier models by a factor of 5x to 10x in the front-end domain.

Limitations and Future Work

While Flame represents a significant step forward, several challenges remain. The current evaluation focuses on single-turn generation; future work will explore multi-turn iterative debugging. Additionally, the framework is currently limited to React and single-page components. Expanding the synthesis pipeline to other frameworks like Vue or Angular and addressing multi-page routing complexity are logical next steps for the project.

Summary updated

Abstract

Modern front-end (FE) development, especially when leveraging the unique features of frameworks like React and Vue, presents distinctive challenges. These include managing modular architectures, ensuring synchronization between data and visual outputs for declarative rendering, and adapting reusable components to various scenarios. Such complexities make it particularly difficult for state-of-the-art large vision-language models (VLMs) to generate accurate and functional code directly from design images. To address these challenges, we propose a reflective agentic workflow that synthesizes high-quality image-text data to capture the diverse characteristics of FE development. This workflow automates the extraction of self-contained\footnote{A \textbf{self-contained} code snippet is one that encapsulates all necessary logic, styling, and dependencies, ensuring it functions independently without requiring external imports or context.} code snippets from real-world projects, renders the corresponding visual outputs, and generates detailed descriptions that link design elements to functional code. To further expand the scope and utility of the synthesis, we introduce three data synthesis strategies: Evolution-based synthesis, which enables scalable and diverse dataset expansion; Waterfall-Model-based synthesis, which generates logically coherent code derived from system requirements; and Additive Development synthesis, which iteratively increases the complexity of human-authored components. We build a large vision-language model, Flame, trained on the synthesized datasets and demonstrate its effectiveness in generating React code via the $\text{pass}@k$ metric. Our results suggest that a code VLM trained to interpret images before code generation may achieve better performance.
BibTeX
@misc{ge2025advancing,
  author = {Tong Ge and Yashu Liu and Jieping Ye and Tianyi Li and Chao Wang},
  title = {{Advancing vision-language models in front-end development via data synthesis}},
  year = {2025},
  eprint = {2503.01619},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2503.01619},
}
Figure