Introduction
Fashion design is inherently a multi-stage process, transitioning from abstract ideation to granular iteration. Designers typically begin by brainstorming high-level "vibes"—such as "elegant" or "youthful"—to establish a design draft. This is followed by a meticulous refinement phase where specific attributes like sleeve length, collar type, or hemline are adjusted.
Despite the rapid advancement of generative AI, existing models often struggle to replicate this professional workflow. Most current systems are "one-shot" generators that conflate high-level concepts and low-level specifications into a single prompt, resulting in limited control. Furthermore, editing specific features often inadvertently alters the global consistency of the garment. HieraFashDiff addresses these challenges by introducing a hierarchical diffusion-based framework that mirrors the human design process.
The HieraFashDiff Framework
HieraFashDiff decomposes the standard reverse diffusion process into two distinct stages: Ideation and Iteration. By splitting the 1,000 denoising steps, the model allows for a natural progression from coarse global structures to fine-grained details.
Stage 1: Ideation
In the early denoising steps, the model is conditioned on high-level design concepts. This stage focuses on establishing the overall silhouette and "feel" of the garment, resulting in a stable design draft.
Stage 2: Iteration and Attribute Ordering
The later denoising steps are dedicated to refining specific apparel attributes. To maximize editability and maintain structural integrity, the researchers implemented a Spatial Impact Strategy. Attributes are ordered based on the size of the area they affect; for instance, "clothing length" is processed before "collar type." This ensures that large-scale structural changes do not overwrite the fine details added in subsequent steps.
Iterative Local Editing
For precise modifications, HieraFashDiff utilizes a pose-conditioned UNet and the Segment Anything Model (SAM). By incorporating 2D pose maps, the model can accurately localize where specific changes should occur—ensuring a "short sleeve" prompt targets the arms rather than the torso. This allows designers to "blend" new attributes into specific regions while preserving the original fabric texture, color, and model pose.
The HieraFashion Dataset
A significant contribution of this work is the curation of the HieraFashion dataset. It contains over 5,000 high-resolution (768x1024) images, each meticulously annotated by professional designers. Unlike previous datasets, HieraFashion provides hierarchical descriptions, separating high-level themes from low-level technical attributes, which is essential for training models to understand the design workflow.
Results and Evaluation
HieraFashDiff demonstrates superior performance compared to state-of-the-art models like Stable Diffusion and Attend-and-Excite.
- Fidelity: The model achieved a Frechet Inception Distance (FID) of 10.27, significantly lower than the 15.33 achieved by the next best model, indicating higher realism.
- Prompt Adherence: A CLIP-S score of 30.61 confirms that generated images align more closely with complex text prompts.
- Qualitative Success: Visually, HieraFashDiff is uniquely capable of performing multiple sequential edits—such as changing a dress to short sleeves and then adding a boat neck—without losing the original garment's identity.
Conclusion and Future Work
HieraFashDiff successfully bridges the gap between AI generation and professional fashion design by treating diffusion as a hierarchical process. While currently limited by a relatively small dataset and a reliance on 2D representations, the framework sets a new standard for controllable fashion AI. Future research will look toward expanding dataset diversity and exploring 3D garment generation based on these hierarchical principles.
Summary updated