RecipeGen: A Benchmark for Real-World Recipe Image Generation

Ruoxuan Zhang, Hongxia Xie, Yi Yao, Jian-Yu Jiang-Lin, Bin Wen, Ling Lo, Hong-Han Shuai, Yung-Hui Li, Wen-Huang ChengarXiv 2025 · DOI · arXiv

Introduction

Generating high-quality images of finished dishes—often referred to as "food porn"—has become a forte of modern Text-to-Image (T2I) models. However, these models frequently falter when tasked with visualizing the process of cooking. RecipeGen (RGB) is a new large-scale benchmark designed to bridge this gap, providing over 139,000 images paired with detailed sequential steps across 21,944 recipes. It represents the first comprehensive dataset specifically curated to train AI in generating realistic, step-by-step culinary illustrations.

The Challenge of Visualizing Culinary Processes

Most existing food datasets, such as Recipe1M, typically provide only a single image of the final meal. While some datasets include intermediate steps, they often rely on "comic-style" illustrations (VGSI) or video frames (YouCook2) where the chef’s hands, kitchen clutter, or poor camera angles obscure the food. This lack of clean, real-world data makes it difficult for T2I models to maintain visual consistency. For instance, a model might struggle to distinguish the visual transformation of beef being pan-fried as a steak versus being braised with vegetables. RecipeGen provides the necessary "ground truth" for these transitions.

Methodology: Curating High-Quality Sequential Data

The authors developed a rigorous pipeline to curate the RGB dataset from user-uploaded content on the platform douguo.com:

  1. Targeted Collection: 158 keywords were used to ensure broad coverage across diverse regions (e.g., Sichuan, Mexican, French), cooking styles (e.g., stir-frying, roasting), and food categories.
  2. Automated Quality Control: GPT-4o was employed to refine raw user data, merging redundant steps, summarizing for coherence, and translating instructions into English.
  3. Filtering and Verification: Recipes with mismatched image-to-step counts or vague instructions were discarded, followed by human spot-checks to ensure logical continuity and dish accuracy.

The RecipeGen Benchmark (RGB)

The primary contribution, the RecipeGen Benchmark (RGB), consists of 21,944 recipes and 139,872 image-step pairs. Unlike previous datasets that lean heavily toward Western cuisine, RGB emphasizes East and Southeast Asian culinary traditions.

Distribution of keywords across region, dessert, cooking style, and staple categories in the RGB dataset.

Compared to video-based or illustrated datasets, RGB provides cleaner, food-focused photography that captures the essence of each preparation stage.

Key Insights: The "Process" Gap and Data Quality

A central finding of this research is that the most significant challenge in recipe generation lies in the transitions between steps. Capturing the visual state of ingredients as they transform is crucial for educational and assistive tools.

A representative image from the RGB dataset showing a clear, food-centric intermediate step.

The authors argue that amateur-contributed photos are often superior to professional videos for training these models. While professional videos may have higher production value, they frequently include "noise" like the chef's body or suboptimal angles. In contrast, amateur photos tend to focus strictly on the food at each stage. Furthermore, the study demonstrates the efficacy of LLMs like GPT-4o as sophisticated data-cleaning tools for transforming messy user comments into structured instructional captions.

Evaluation and Results

The dataset was evaluated using Goal Faithfulness (GF) and Step Faithfulness (SF), utilizing CLIP scores to measure alignment between images and text. RGB achieved a GF score of 85.40 and an SF score of 84.44, significantly outperforming the VGSI-Recipe dataset (81.40 and 73.70, respectively). Most recipes in the dataset contain over six steps and an average of nine ingredients, providing the depth required for models to learn complex visual interactions and long-range semantic relationships.

Conclusion and Future Work

While the use of GPT-4o and real-world data introduces some variability and the potential for minor hallucinations, RecipeGen serves as a foundational resource for the next generation of T2I models. Future work will focus on leveraging this benchmark to improve "visual character consistency," ensuring that ingredients maintain a consistent appearance from the first step of preparation to the final plating.

Summary updated

Abstract

Recipe image generation is an important challenge in food computing, with applications from culinary education to interactive recipe platforms. However, there is currently no real-world dataset that comprehensively connects recipe goals, sequential steps, and corresponding images. To address this, we introduce RecipeGen, the first real-world goal-step-image benchmark for recipe generation, featuring diverse ingredients, varied recipe steps, multiple cooking styles, and a broad collection of food categories. Data is in https://github.com/zhangdaxia22/RecipeGen.
BibTeX
@misc{zhang2025recipegen,
  author = {Ruoxuan Zhang and Hongxia Xie and Yi Yao and Jian-Yu Jiang-Lin and Bin Wen and Ling Lo and Hong-Han Shuai and Yung-Hui Li and Wen-Huang Cheng},
  title = {{RecipeGen: A Benchmark for Real-World Recipe Image Generation}},
  year = {2025},
  eprint = {2503.05228},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2503.05228},
}
Figure