Abstract
This paper introduces FLORA, the first comprehensive dataset of 4,330 fashion sketches paired with professional-grade textual descriptions, and NeRA, a novel adapter architecture. By replacing the standard linear transformations in Parameter-Efficient Fine-Tuning (PEFT) with non-linear Kolmogorov-Arnold Networks (KANs), the authors achieve superior semantic alignment and faster convergence in generating high-fidelity fashion outfits from text.
Introduction: The "Last Mile" of Fashion AI
While AI has made massive strides in general image generation, the fashion industry presents a unique "last mile" challenge. Professional fashion design relies on sketches—abstract, stylistically rich representations—rather than photorealistic images. Existing models struggle here for two primary reasons:
- Data Scarcity: There are few datasets that pair fashion sketches with the specific, technical jargon used by designers (e.g., "asymmetrical neckline," "tiered ruffles," or "double-breasted waistcoat").
- Architectural Rigidity: Popular fine-tuning methods like LoRA (Low-Rank Adaptation) rely on linear transformations. However, the relationship between a complex textual description and a nuanced visual style is inherently non-linear. Standard adapters often fail to capture the subtle stylistic elements required for high-end design.
The FLORA Dataset Pipeline
To address the data gap, the authors developed FLORA (Fashion Language Outfit Representation for Apparel Generation). This dataset consists of 4,330 curated pairs of fashion sketches and technical descriptions. The creation process involved a rigorous multi-stage pipeline to ensure high quality and professional relevance.
The pipeline includes several critical steps:
- VLM Filtering: Using LLaVa to ensure images contained only a single outfit sketch, filtering out noise from the initial scrape of 10,000 images.
- Automated Cleanup: Employing Keras-OCR and ControlNet to remove watermarks and signatures without damaging the underlying sketches.
- Expert Annotation: Leveraging GPT-4o to generate descriptions structured around model pose, outfit details, color, and accessories, specifically using industry-standard terminology.
The NeRA Adapter Architecture
The NeRA (Nonlinear low-rank Expressive Representation Adapter) is the paper's technical centerpiece. Most adapters (like LoRA or DoRA) use Multi-Layer Perceptrons (MLPs) with fixed activation functions. NeRA instead utilizes Kolmogorov-Arnold Networks (KANs).
Unlike MLPs, which have weights on edges, KANs have learnable, non-linear activation functions (splines or Radial Basis Functions) on the edges. This allows NeRA to model complex, high-order relationships between text and pixels more efficiently than linear methods. It acts as a "drop-in" replacement for MLP layers in large diffusion models like FLUX.1 or Stable Diffusion 3.
Key Insights and Results
The research highlights that non-linearity is essential for creativity. Fashion design isn't a linear mapping; for instance, a "mermaid gown" involves specific geometric constraints that linear adapters struggle to maintain when other variables change. NeRA’s spline-based transformations handle these dependencies much better.
Performance Benchmarks
The authors tested NeRA against state-of-the-art adapters (LoRA, LoKR, DoRA, LoHA) across multiple base models:
- Quantitative Superiority: On the FLUX model, NeRA achieved an FID (Fréchet Inception Distance) of 6.05, significantly beating LoRA’s 7.88. Lower FID indicates higher image realism and diversity.
- Semantic Alignment: NeRA scored higher on CLIP-SIM (0.3412 vs LoRA’s 0.2987), proving that the generated images matched the complex prompts more accurately.
- Efficiency: Despite its complexity, NeRA reached optimal performance in significantly fewer training steps than LoRA, reducing total training time by nearly half in some configurations.
In a blind human evaluation with 500 participants, NeRA-generated sketches were rated higher in Creativity, Prompt Relevance, and Visual Quality compared to all other PEFT methods.
Conclusion and Future Directions
While a major step forward, the authors identify several areas for future work. Currently, the model generates the whole outfit; future iterations aim to allow "part-level control" for editing specific elements like sleeves or necklines. Additionally, the team plans to expand the FLORA dataset to include more diverse cultural styles and optimize KAN-based inference for real-time mobile applications.
Summary updated