Abstract
Image virtual try-on task has abundant applications and has become a hot research topic recently. Existing 2D image-based virtual try-on methods aim to transfer a target clothing image onto a reference person, which has two main disadvantages: cannot control the size and length precisely; unable to accurately estimate the user's figure in the case of users wearing thick clothes, resulting in inaccurate dressing effect. In this paper, we put forward an akin task that aims to dress clothing for underwear models. To solve the above drawbacks, we propose a Shape Controllable Virtual Try-On Network (SC-VTON), where a graph attention network integrates the information of model and clothing to generate the warped clothing image. In addition, the control points are incorporated into SC-VTON for the desired clothing shape. Furthermore, by adding a Splitting Network and a Synthesis Network, we can use clothing/model pair data to help optimize the deformation module and generalize the task to the typical virtual try-on task. Extensive experiments show that the proposed method can achieve accurate shape control. Meanwhile, compared with other methods, our method can generate high-resolution results with detailed textures.
Introduction
Virtual try-on (VTON) technology is a cornerstone of modern e-commerce, allowing consumers to visualize garments on their own bodies. While 3D-based methods provide high fidelity and control, their computational cost and the requirement for specialized hardware make them difficult to scale. Consequently, 2D image-based methods have gained popularity. However, these methods often lack the ability to precisely adjust clothing parameters like sleeve length or waist tightness. Furthermore, they struggle with "baggy clothes bias"—the inability to accurately estimate a user's true body shape when they are wearing loose-fitting garments in the reference photo.
This paper introduces SC-VTON (Shape Controllable Virtual Try-On Network), a framework presented at ACM Multimedia 2021 designed to overcome these limitations. By representing clothing as a deformable graph and utilizing a specialized training strategy focused on "underwear models," SC-VTON enables continuous control over clothing shape while maintaining high-resolution texture details.
Methodology
The SC-VTON architecture consists of two primary stages: a deformation stage based on Graph Attention Networks (GAT) and a self-loop optimization stage for real-world generalization.
GAT-based Deformation and Shape Control
Traditional 2D VTON methods often rely on Thin-Plate Splines (TPS) for clothing warping. However, TPS is a global transformation that can lead to unrealistic texture distortions. SC-VTON instead treats the garment as a mesh graph. A Graph Attention Network (GAT) is employed to predict coordinate offsets for every vertex in this mesh. This localized approach allows for more natural deformations that preserve the garment's original silhouette and intricate patterns.
To enable user interaction, the system incorporates control points near the model's silhouette. By manipulating these points—for instance, moving them along the limbs—users can adjust the length and tightness of the garment in a continuous fashion.
Self-loop Optimization
A significant challenge in VTON research is the lack of paired "ground truth" data (the same person in both underwear and the target clothing). To address this, the authors propose a "self-loop" mechanism involving a Splitting Network and a Synthesis Network. The Splitting Network is trained to "remove" clothing from a person to estimate their underlying figure, while the Synthesis Network recombines this estimated figure with the warped clothing. This allows the model to be trained effectively on standard fashion datasets.
Experiments and Results
The model was evaluated using two primary datasets: the newly introduced UMV (Underwear Model Virtual Try-On) dataset, which contains over 140,000 pseudo-labeled pairs, and the standard MPV (Multi-Pose Virtual Try-On) benchmark.
Qualitative Performance
Compared to state-of-the-art models like ACGPN, SC-VTON produces significantly sharper results. It successfully avoids common artifacts, such as the blending of sleeves into the torso, and maintains the integrity of complex textures even under significant deformation.
User Evaluation
The effectiveness of SC-VTON was further validated through a user study on Alibaba's "Qince" platform. Participants consistently preferred SC-VTON's results over competing methods, particularly at higher resolutions (512x384), where the model's ability to preserve fine details was most apparent. Ablation studies confirmed that the iterative GAT refinement and the inclusion of clothing-specific information were critical to achieving high vertex prediction precision.
Conclusion
SC-VTON represents a significant step forward in 2D virtual try-on technology by introducing precise, continuous shape control. By leveraging graph-based deformation and training on underwear models, the framework provides a more realistic and interactive user experience. Future work will focus on improving performance for side-view poses and handling complex occlusions where body parts cross in front of the garment.
Summary updated