Reference-Free Image Quality Assessment for Virtual Try-On via Human Feedback

Yuki Hirakawa, Takashi Wada, Ryotaro Shimizu, Takuya Furusawa, Yuki Saito, Ryosuke Araki, Tianwei Chen, Fan Mo, Yoshimitsu Aoki

Abstract

Given a person image and a garment image, image-based Virtual Try-ON (VTON) synthesizes a try-on image of the person wearing the target garment. As VTON systems become increasingly important in practical applications such as fashion e-commerce, reliable evaluation of their outputs has emerged as a critical challenge. In real-world scenarios, ground-truth images of the same person wearing the target garment are typically unavailable, making reference-based evaluation impractical. Moreover, widely used distribution-level metrics such as Fréchet Inception Distance and Kernel Inception Distance measure dataset-level similarity and fail to reflect the perceptual quality of individual generated images. To address these limitations, we propose Image Quality Assessment for Virtual Try-On (VTON-IQA), a reference-free framework for human-aligned, image-level quality assessment without requiring ground-truth images. To model human perceptual judgments, we construct VTON-QBench, a large-scale human-annotated benchmark comprising 62,688 try-on images generated by 14 representative VTON models and 431,800 quality annotations collected from 13,838 qualified annotators. To the best of our knowledge, this is the largest dataset to date for human subjective evaluation in virtual try-on. Evaluating virtual try-on quality requires verifying both garment fidelity and the preservation of person-specific details. To explicitly model such interactions, we introduce an Interleaved Cross-Attention module that extends standard transformer blocks by inserting a cross-attention layer between self-attention and MLP in the latter blocks. Extensive experiments show that VTON-IQA achieves reliable human-aligned image-level quality prediction. Moreover, we conduct a comprehensive benchmark evaluation of 14 representative VTON models using VTON-IQA.

Introduction

Virtual Try-On (VTON) technology aims to synthesize a realistic image of a person wearing a specific garment. While this technology is rapidly expanding in fashion e-commerce, evaluating the quality of these generated images remains a significant hurdle. Traditional metrics like SSIM or LPIPS are reference-based, requiring a ground-truth image of the person wearing the specific garment—a luxury rarely available in real-world applications. Distribution-level metrics like FID fail to assess individual image quality, often missing perceptual artifacts that a human would immediately notice.

Examples of garment and person pairs used in virtual try-on tasks.

To bridge this gap, we propose VTON-IQA, a reference-free framework designed for human-aligned, image-level quality assessment. By explicitly modeling the interactions between the garment, the person, and the generated output, VTON-IQA provides a reliable measure of quality without needing a reference photo.

The VTON-IQA Framework

The VTON-IQA architecture utilizes a three-branch transformer to process the garment image ($I_G$), the person image ($I_P$), and the generated try-on image ($I_V$). The core innovation is the Interleaved Cross-Attention (ICA) module. Unlike standard symmetric transformers, ICA treats the generated image as a central hub. In the network's latter blocks, cross-attention layers are inserted between self-attention and MLP layers to model asymmetric dependencies: the generated image must maintain fidelity to the garment while preserving the person's identity and background.

The model is trained using a joint loss function that combines pairwise preference learning (based on the Bradley-Terry model) and absolute score regression. This dual approach allows the model to capture both relative quality differences and absolute perceptual ratings, aligning more closely with how humans evaluate visual fidelity.

VTON-QBench: A Large-Scale Human-Annotated Benchmark

To train and validate VTON-IQA, we constructed VTON-QBench, the largest dataset to date for human subjective evaluation in virtual try-on. It contains 62,688 try-on images generated by 14 representative VTON models, supported by 431,800 quality annotations from over 13,000 qualified annotators.

The five-step pipeline for constructing VTON-QBench, including synthetic data augmentation and rigorous human annotation.

As shown in the pipeline above, we addressed the limited diversity of existing datasets by using FLUX.1-dev to generate synthetic garment-person pairs across diverse fashion styles. A rigorous two-stage curation process, involving sanity checks and inter-annotator agreement metrics (Krippendorff’s alpha), ensured the high quality of the collected feedback.

Results and Model Benchmarking

Extensive experiments demonstrate that VTON-IQA achieves superior alignment with human judgment compared to traditional and zero-shot metrics. It reached a Spearman correlation ($ ho_{SRCC}$) of 0.750, significantly outperforming LPIPS and DINOv3-based baselines. In pairwise consistency tests, the model achieved 79% accuracy, nearly matching the 79.1% consistency observed between different groups of human annotators.

Using VTON-IQA to benchmark 14 major VTON models revealed that proprietary image-editing models currently lead in human-perceived quality, often surpassing specialized academic U-Net or DiT-based architectures. The framework also proved robust to global image shifts (like pose or zoom) that typically skew traditional reference-based metrics, focusing instead on perceptual artifacts and fidelity.

Conclusion

VTON-IQA provides a robust, reference-free solution for evaluating virtual try-on systems. While currently focused on studio-style 2D images, future work will aim to improve interpretability—providing specific reasons for quality scores—and extend the framework to video-based VTON to account for temporal consistency. By aligning automated metrics with human perception, VTON-IQA paves the way for more realistic and reliable virtual fashion experiences.

Summary updated

Abstract

Given a person image and a garment image, image-based Virtual Try-ON (VTON) synthesizes a try-on image of the person wearing the target garment. As VTON systems become increasingly important in practical applications such as fashion e-commerce, reliable evaluation of their outputs has emerged as a critical challenge. In real-world scenarios, ground-truth images of the same person wearing the target garment are typically unavailable, making reference-based evaluation impractical. Moreover, widely used distribution-level metrics such as Fréchet Inception Distance and Kernel Inception Distance measure dataset-level similarity and fail to reflect the perceptual quality of individual generated images. To address these limitations, we propose Image Quality Assessment for Virtual Try-On (VTON-IQA), a reference-free framework for human-aligned, image-level quality assessment without requiring ground-truth images. To model human perceptual judgments, we construct VTON-QBench, a large-scale human-annotated benchmark comprising 62,688 try-on images generated by 14 representative VTON models and 431,800 quality annotations collected from 13,838 qualified annotators. To the best of our knowledge, this is the largest dataset to date for human subjective evaluation in virtual try-on. Evaluating virtual try-on quality requires verifying both garment fidelity and the preservation of person-specific details. To explicitly model such interactions, we introduce an Interleaved Cross-Attention module that extends standard transformer blocks by inserting a cross-attention layer between self-attention and MLP in the latter blocks. Extensive experiments show that VTON-IQA achieves reliable human-aligned image-level quality prediction. Moreover, we conduct a comprehensive benchmark evaluation of 14 representative VTON models using VTON-IQA.
BibTeX
@misc{hirakawa2026reference,
  author = {Yuki Hirakawa and Takashi Wada and Ryotaro Shimizu and Takuya Furusawa and Yuki Saito and Ryosuke Araki and Tianwei Chen and Fan Mo and Yoshimitsu Aoki},
  title = {{Reference-Free Image Quality Assessment for Virtual Try-On via Human Feedback}},
  year = {2026},
  eprint = {2603.13057},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2603.13057},
}
Figure