Introduction
Generating high-quality images of finished dishes—often referred to as "food porn"—has become a forte of modern Text-to-Image (T2I) models. However, these models frequently falter when tasked with visualizing the process of cooking. RecipeGen (RGB) is a new large-scale benchmark designed to bridge this gap, providing over 139,000 images paired with detailed sequential steps across 21,944 recipes. It represents the first comprehensive dataset specifically curated to train AI in generating realistic, step-by-step culinary illustrations.
The Challenge of Visualizing Culinary Processes
Most existing food datasets, such as Recipe1M, typically provide only a single image of the final meal. While some datasets include intermediate steps, they often rely on "comic-style" illustrations (VGSI) or video frames (YouCook2) where the chef’s hands, kitchen clutter, or poor camera angles obscure the food. This lack of clean, real-world data makes it difficult for T2I models to maintain visual consistency. For instance, a model might struggle to distinguish the visual transformation of beef being pan-fried as a steak versus being braised with vegetables. RecipeGen provides the necessary "ground truth" for these transitions.
Methodology: Curating High-Quality Sequential Data
The authors developed a rigorous pipeline to curate the RGB dataset from user-uploaded content on the platform douguo.com:
- Targeted Collection: 158 keywords were used to ensure broad coverage across diverse regions (e.g., Sichuan, Mexican, French), cooking styles (e.g., stir-frying, roasting), and food categories.
- Automated Quality Control: GPT-4o was employed to refine raw user data, merging redundant steps, summarizing for coherence, and translating instructions into English.
- Filtering and Verification: Recipes with mismatched image-to-step counts or vague instructions were discarded, followed by human spot-checks to ensure logical continuity and dish accuracy.
The RecipeGen Benchmark (RGB)
The primary contribution, the RecipeGen Benchmark (RGB), consists of 21,944 recipes and 139,872 image-step pairs. Unlike previous datasets that lean heavily toward Western cuisine, RGB emphasizes East and Southeast Asian culinary traditions.
Compared to video-based or illustrated datasets, RGB provides cleaner, food-focused photography that captures the essence of each preparation stage.
Key Insights: The "Process" Gap and Data Quality
A central finding of this research is that the most significant challenge in recipe generation lies in the transitions between steps. Capturing the visual state of ingredients as they transform is crucial for educational and assistive tools.
The authors argue that amateur-contributed photos are often superior to professional videos for training these models. While professional videos may have higher production value, they frequently include "noise" like the chef's body or suboptimal angles. In contrast, amateur photos tend to focus strictly on the food at each stage. Furthermore, the study demonstrates the efficacy of LLMs like GPT-4o as sophisticated data-cleaning tools for transforming messy user comments into structured instructional captions.
Evaluation and Results
The dataset was evaluated using Goal Faithfulness (GF) and Step Faithfulness (SF), utilizing CLIP scores to measure alignment between images and text. RGB achieved a GF score of 85.40 and an SF score of 84.44, significantly outperforming the VGSI-Recipe dataset (81.40 and 73.70, respectively). Most recipes in the dataset contain over six steps and an average of nine ingredients, providing the depth required for models to learn complex visual interactions and long-range semantic relationships.
Conclusion and Future Work
While the use of GPT-4o and real-world data introduces some variability and the potential for minor hallucinations, RecipeGen serves as a foundational resource for the next generation of T2I models. Future work will focus on leveraging this benchmark to improve "visual character consistency," ensuring that ingredients maintain a consistent appearance from the first step of preparation to the final plating.
Summary updated