Introduction
KiseKloset is an end-to-end fashion AI system designed to act as a "digital sales assistant." It integrates visual and textual item retrieval, a generalized transformer-based outfit recommender, and real-time virtual try-on (VTON) into a single, cohesive pipeline. By optimizing for speed with approximate search algorithms and lightweight neural architectures, the system provides a seamless user experience that significantly improves satisfaction in e-commerce environments.
Addressing the Fragmented Shopping Experience
Online fashion shopping is often a fragmented process. Users frequently find an item they like (retrieval) but struggle to find matching pieces (recommendation) or visualize how the complete outfit looks on their body (try-on). Existing research often treats these as isolated problems. Furthermore, high-quality VTON models are typically too computationally expensive for real-time web applications, and recommendation models are often limited to simple "fill-in-the-blank" tasks. KiseKloset bridges these gaps with a unified, efficient system.
Core Methodologies
The KiseKloset system is built on three main pillars:
1. Multi-Modal Retrieval
The system supports both intra-category search using CLIP embeddings to find visually similar items and text-guided search via the CLIP4Cir architecture. This allows users to refine their searches with natural language (e.g., "same material but in blue"). To ensure sub-millisecond query times across large datasets, the authors integrated the HNSW (Hierarchical Navigable Small Worlds) algorithm for Approximate Nearest Neighbor (ANN) search.
2. Generalized Outfit Recommendation
KiseKloset utilizes a novel Transformer-based encoder for inter-category recommendations. Unlike traditional models, this architecture employs special learnable tokens—<OUT> for suggested categories and <UN> for unavailable ones—allowing it to generate a complete, multi-item outfit from a single reference item. The system supports two distinct stylistic modes: "Tone sur tone" (matching colors and textures) and "Mix and match" (diverse, "hipster" styles).
3. Real-Time Virtual Try-On (VTON)
For the try-on component, the system employs DM-VTON (Distilled Mobile Real-time Virtual Try-On). This is a parser-free approach, meaning it does not require pre-calculating complex human body maps during inference. Through a teacher-student distillation process, a lightweight "student" model is trained to produce realistic results at high speeds suitable for browser-based applications.
Evaluation and Performance
The system was trained and evaluated using industry-standard benchmarks, including FashionIQ for text-guided retrieval, PolyvoreOutfits for recommendation logic, and DeepFashion for large-scale search benchmarking.
Key results include:
- Search Efficiency: The HNSW algorithm achieved a query time of 1.80ms, significantly outperforming standard KNN (35.24ms) while maintaining a 98.11% recall rate.
- User Satisfaction: In a user study, 84% of participants found the system highly useful. Qualitative scores (on a 1–5 scale) highlighted "Response Time" (4.3) and "Ease of Use" (4.1) as major strengths.
- Stylistic Variety: To prevent repetitive results, the authors implemented a tiered retrieval strategy that mixes accurate, approximate, and heuristic results, increasing user engagement through variety.
Conclusion and Future Work
While KiseKloset currently focuses on upper-body garments, it demonstrates the viability of a unified fashion AI system. Future work will involve expanding the recommendation and try-on modules to include lower-body items and full-body dresses. Additionally, the authors aim to address current VTON limitations, such as artifacts in complex poses and unrealistic scaling due to a lack of specific sizing data.
Summary updated