KiseKloset: Comprehensive System For Outfit Retrieval, Recommendation, And Try-On

Thanh-Tung Phan-Nguyen, Khoi-Nguyen Nguyen-Ngoc, Tam V. Nguyen, Minh-Triet Tran, Trung-Nghia LearXiv 2025 · DOI · arXiv

Introduction

KiseKloset is an end-to-end fashion AI system designed to act as a "digital sales assistant." It integrates visual and textual item retrieval, a generalized transformer-based outfit recommender, and real-time virtual try-on (VTON) into a single, cohesive pipeline. By optimizing for speed with approximate search algorithms and lightweight neural architectures, the system provides a seamless user experience that significantly improves satisfaction in e-commerce environments.

The KiseKloset user interface, integrating model selection, garment retrieval, and real-time virtual try-on results.

Addressing the Fragmented Shopping Experience

Online fashion shopping is often a fragmented process. Users frequently find an item they like (retrieval) but struggle to find matching pieces (recommendation) or visualize how the complete outfit looks on their body (try-on). Existing research often treats these as isolated problems. Furthermore, high-quality VTON models are typically too computationally expensive for real-time web applications, and recommendation models are often limited to simple "fill-in-the-blank" tasks. KiseKloset bridges these gaps with a unified, efficient system.

Core Methodologies

The KiseKloset system is built on three main pillars:

1. Multi-Modal Retrieval

The system supports both intra-category search using CLIP embeddings to find visually similar items and text-guided search via the CLIP4Cir architecture. This allows users to refine their searches with natural language (e.g., "same material but in blue"). To ensure sub-millisecond query times across large datasets, the authors integrated the HNSW (Hierarchical Navigable Small Worlds) algorithm for Approximate Nearest Neighbor (ANN) search.

2. Generalized Outfit Recommendation

KiseKloset utilizes a novel Transformer-based encoder for inter-category recommendations. Unlike traditional models, this architecture employs special learnable tokens—<OUT> for suggested categories and <UN> for unavailable ones—allowing it to generate a complete, multi-item outfit from a single reference item. The system supports two distinct stylistic modes: "Tone sur tone" (matching colors and textures) and "Mix and match" (diverse, "hipster" styles).

An example of 'Tone sur tone' recommendation, where the system suggests accessories and footwear that match the color and texture of the primary garment.

3. Real-Time Virtual Try-On (VTON)

For the try-on component, the system employs DM-VTON (Distilled Mobile Real-time Virtual Try-On). This is a parser-free approach, meaning it does not require pre-calculating complex human body maps during inference. Through a teacher-student distillation process, a lightweight "student" model is trained to produce realistic results at high speeds suitable for browser-based applications.

Evaluation and Performance

The system was trained and evaluated using industry-standard benchmarks, including FashionIQ for text-guided retrieval, PolyvoreOutfits for recommendation logic, and DeepFashion for large-scale search benchmarking.

Key results include:

  • Search Efficiency: The HNSW algorithm achieved a query time of 1.80ms, significantly outperforming standard KNN (35.24ms) while maintaining a 98.11% recall rate.
  • User Satisfaction: In a user study, 84% of participants found the system highly useful. Qualitative scores (on a 1–5 scale) highlighted "Response Time" (4.3) and "Ease of Use" (4.1) as major strengths.
  • Stylistic Variety: To prevent repetitive results, the authors implemented a tiered retrieval strategy that mixes accurate, approximate, and heuristic results, increasing user engagement through variety.

Conclusion and Future Work

While KiseKloset currently focuses on upper-body garments, it demonstrates the viability of a unified fashion AI system. Future work will involve expanding the recommendation and try-on modules to include lower-body items and full-body dresses. Additionally, the authors aim to address current VTON limitations, such as artifacts in complex poses and unrealistic scaling due to a lack of specific sizing data.

Summary updated

Abstract

The global fashion e-commerce industry has become integral to people's daily lives, leveraging technological advancements to offer personalized shopping experiences, primarily through recommendation systems that enhance customer engagement through personalized suggestions. To improve customers' experience in online shopping, we propose a novel comprehensive KiseKloset system for outfit retrieval and recommendation. We explore two approaches for outfit retrieval: similar item retrieval and text feedback-guided item retrieval. Notably, we introduce a novel transformer architecture designed to recommend complementary items from diverse categories. Furthermore, we enhance the overall performance of the search pipeline by integrating approximate algorithms to optimize the search process. Additionally, addressing the crucial needs of online shoppers, we employ a lightweight yet efficient virtual try-on framework capable of real-time operation, memory efficiency, and maintaining realistic outputs compared to its predecessors. This virtual try-on module empowers users to visualize specific garments on themselves, enhancing the customers' experience and reducing costs associated with damaged items for retailers. We deployed our end-to-end system for online users to test and provide feedback, enabling us to measure their satisfaction levels. The results of our user study revealed that 84% of participants found our comprehensive system highly useful, significantly improving their online shopping experience.
BibTeX
@misc{phannguyen2025kisekloset,
  author = {Thanh-Tung Phan-Nguyen and Khoi-Nguyen Nguyen-Ngoc and Tam V. Nguyen and Minh-Triet Tran and Trung-Nghia Le},
  title = {{KiseKloset: Comprehensive System For Outfit Retrieval, Recommendation, And Try-On}},
  year = {2025},
  eprint = {2506.23471},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2506.23471},
}
Figure