FashionReGen: LLM-Empowered Fashion Report Generation

Yujuan Ding, Yunshan Ma, Wenqi Fan, Yige Yao, Tat-Seng Chua, Qing Li

TL;DR

FashionReGen introduces GPT-FAR, a first-of-its-kind automated pipeline that transforms raw catwalk images into professional-grade fashion analysis reports. By leveraging the multimodal capabilities of GPT-4V alongside a novel iterative tag-cleaning process, the system performs fine-grained garment recognition, statistical trend analysis, and expert-level prose generation.

What Problem Does This Paper Address?

The fashion industry moves at breakneck speed, making trend forecasting essential for brands and consumers. Traditionally, fashion reports are produced manually by high-priced experts who review thousands of runway looks to identify shifts in styles, silhouettes, and fabrics.

This manual process is not only expensive and slow but also prone to the subjective biases of a small group of individuals. Automating this—a task the authors call Fashion Report Generation (FashionReGen)—is incredibly difficult because it requires:

  1. Fine-grained visual recognition: Distinguishing between subtle attribute details (e.g., a "draped neckline" vs. a "cowl neck").
  2. Domain expertise: Applying professional statistical metrics to fashion data.
  3. Multimodal synthesis: Coherently blending text, data charts, and images into a professional layout.

Methods

The authors propose the GPT-FAR (Fashion Analyzing and Reporting) system, which operates in three distinct stages:

  1. Catwalk Understanding:
    • Garment Tagging: Uses GPT-4V to identify categories (dresses, skirts, etc.) and attributes (silhouette, neckline, fabric) based on professional methodologies (specifically referencing WGSN standards).
  2. Two-Stage Tag Cleaning: Because LLMs often produce inconsistent labels (e.g., "draped" vs. "draping"), the authors developed a "manual format unification" step followed by an automatic synonym emergence process. This uses GPT-4 iteratively to group synonymous tags into a unified dictionary, making the data "clean" enough for math.
  3. Collective Organization and Analysis: The system aggregates individual garment tags into collection-level data. It calculates professional metrics like Mix (the proportion of a specific style), Year-on-Year (YoY) index (growth or decline of a trend), and Evolving Trend lists.

  4. Fashion Report Generation:

    • Visuals: The system automatically plots statistical charts and retrieves representative images.
  5. Textual Analysis: GPT-4V generates professional prose by "reading" the generated charts through in-context learning. It produces both high-level summaries and category-specific deep dives.

Datasets

The researchers evaluated the system using a preliminary dataset of catwalk images from 2019–2023 (5 years).

  • Brands included: Chanel, Saint Laurent, Louis Vuitton, Valentino, Dior, and Givenchy.
  • Focus: Spring/Summer (SS) seasons.
  • Expandability: The authors note the system is designed to scale to more brands and "off-season" collections (Pre-fall, Cruise, etc.).

Key Insights and Ideas

  • The "Structured Data" Gap: A major insight is that while GPT-4V is excellent at describing a single image, its "open-ended" nature is a bug, not a feature, for statistical analysis. The iterative synonym grouping is a clever workaround to turn "vibes" into "variables" that can be graphed.
  • In-Context Learning for Data Interpretation: Rather than just asking an LLM to "write a report," the authors provide the LLM with specific data points from the charts and examples of professional fashion writing. This "grounds" the model, preventing hallucinations and ensuring the tone matches industry standards.
  • Catwalks as Lead Indicators: The system focuses on catwalks because they represent the "top of the funnel" for the fashion industry, where high-end brands set the stage for downstream mass-market trends.

Key Results

  • Qualitative Success: The system successfully generated multi-page reports featuring a cover page, overall analysis (with a three-column layout), and category-specific breakdowns.
  • Professional Prose: The generated text was found to be "reasonable, fluent, and professional," mimicking the style of industry leaders. For example, the system could identify that "Dresses and Skirts marked their territory as frontrunners... accounting for over a third of the market."
  • Efficiency: The pipeline offers a significant reduction in the labor and intellectual cost required to synthesize thousands of runway looks into a coherent narrative.

Limitations and Future Work

  • Data Diversity: The current demo relies on a relatively small dataset of high-fashion brands. The authors plan to incorporate social media, forums, and business/sales data to provide a more holistic view of the market.
  • Evaluation Metrics: One of the biggest hurdles in FashionReGen is how to objectively "score" a report. Future work will focus on developing technical challenges to evaluate the degree of automation and the "intelligence" of the insights.
  • Expanding Report Types: The authors aim to move beyond seasonal summaries into more specialized forecasting reports.

Summary updated

Abstract

Fashion analysis refers to the process of examining and evaluating trends, styles, and elements within the fashion industry to understand and interpret its current state, generating fashion reports. It is traditionally performed by fashion professionals based on their expertise and experience, which requires high labour cost and may also produce biased results for relying heavily on a small group of people. In this paper, to tackle the Fashion Report Generation (FashionReGen) task, we propose an intelligent Fashion Analyzing and Reporting system based the advanced Large Language Models (LLMs), debbed as GPT-FAR. Specifically, it tries to deliver FashionReGen based on effective catwalk analysis, the proposed GPT-FAR system is equipped with several key procedures, namely, catwalk understanding, collective organization and analysis, and report generation. By posing and exploring such an open-ended, complex and domain-specific task of FashionReGen, it is able to test the general capability of LLMs in fashion domain. It also inspires the explorations of more high-level tasks with industrial significance in other domains. Video illustration and more materials of GPT-FAR can be found in https://github.com/CompFashion/FashionReGen.
BibTeX
@inproceedings{ding2024fashionregen,
  author = {Yujuan Ding and Yunshan Ma and Wenqi Fan and Yige Yao and Tat-Seng Chua and Qing Li},
  title = {{FashionReGen: LLM-Empowered Fashion Report Generation}},
  booktitle = {The Web Conference},
  year = {2024},
  doi = {10.1145/3589335.3651232},
}

Loading related content...

Figure