Abstract
Virtual Try-On (VTON) systems often struggle to maintain the high-fidelity details of target garments, such as complex textures and logos, due to misaligned attention in diffusion models. This paper introduces a novel Clothing Region of Interest Self-Correction (CSC) module. CSC is a training-free, plug-and-play approach that uses an energy-guided function to refine attention maps during the denoising process, ensuring the model focuses precisely on the clothing region. Additionally, the authors propose the Virtual Try-on Inception Distance (VTID) to better evaluate garment preservation. Experimental results on VITON-HD and DressCode datasets demonstrate that CSC significantly improves texture fidelity and boundary accuracy, outperforming existing state-of-the-art methods.
Introduction
The goal of image-based Virtual Try-On (VTON) is to synthesize a realistic image of a person wearing a specific target garment. While diffusion-based models have advanced the realism of generated images, they frequently suffer from "fidelity loss." Common issues include the blurring of intricate patterns, the disappearance of brand logos, and "bleeding" where clothing textures spill over onto the person's skin or background. These failures often stem from the model's inability to precisely localize its attention on the Clothing Region of Interest (C-RoI).
Methodology: Clothing Region of Interest Self-Correction (CSC)
The proposed CSC module operates during the inference phase of a pre-trained diffusion model, requiring no additional training. It leverages the observation that certain cross-attention maps in the diffusion process naturally correspond to the spatial layout of the garment.
Energy-Guided Attention Refinement
CSC introduces an energy-guided function that measures the discrepancy between the model's current attention maps and the target clothing mask. This function drives two primary mechanisms:
- Attention Attract: This mechanism pulls scattered attention weights from outside the C-RoI into the correct spatial boundaries.
- Attention Repel: To prevent the attention from becoming overly uniform (which leads to blurred textures), this mechanism ensures high-density attention is maintained on detailed areas of the garment.
By applying the gradient of this energy function to "nudge" the predicted noise at each denoising step, the model is forced to maintain high fidelity to the target garment's structure and texture.
Virtual Try-on Inception Distance (VTID)
Standard metrics like FID measure general image quality but fail to capture how well a specific garment is preserved. The authors introduce VTID, which utilizes human parsing and appearance flow to quantify the similarity between the generated clothing and the target garment, providing a more task-relevant evaluation.
Experiments and Results
The CSC module was evaluated using the VITON-HD and DressCode datasets. When integrated with the OOTDiffusion baseline, CSC achieved state-of-the-art performance across multiple metrics.
Quantitative Performance
On the VITON-HD dataset, the method improved LPIPS by 1.4%, FID by 2.3%, and KID by 12.3%. Notably, the proposed VTID metric showed a 5.8% improvement, confirming superior garment preservation.
Qualitative Analysis
Visual comparisons demonstrate that CSC effectively preserves complex graphics and sharp boundaries that baseline models often miss.
Downstream Application: Cloth-Changing Re-ID
To test real-world utility, the authors used CSC-enhanced images to augment training data for Cloth-Changing Person Re-identification (CC-ReID). This approach improved Rank-1 accuracy by up to 2.5% on the LTCC dataset, showing that higher-fidelity VTON can directly benefit other computer vision tasks.
Conclusion
The CSC module provides a powerful, training-free solution for improving the fidelity of virtual try-on systems. By "policing" attention maps during inference, it ensures that diffusion models respect the boundaries and details of target garments. While the iterative backpropagation adds some computational overhead, the significant gains in texture preservation and its compatibility with existing models make it a valuable tool for high-fidelity image synthesis.
Summary updated