Abstract
Attribute editing has become an important and emerging topic of computer vision. In this paper, we consider a task: given a reference garment image A and another image B with target attribute (collar/sleeve), generate a photo-realistic image which combines the texture from reference A and the new attribute from reference B. The highly convoluted attributes and the lack of paired data are the main challenges to the task. To overcome those limitations, we propose a novel self-supervised model to synthesize garment images with disentangled attributes (e.g., collar and sleeves) without paired data. Our method consists of a reconstruction learning step and an adversarial learning step. The model learns texture and location information through reconstruction learning. And, the model's capability is generalized to achieve single-attribute manipulation by adversarial learning. Meanwhile, we compose a new dataset, named GarmentSet, with annotation of landmarks of collars and sleeves on clean garment images. Extensive experiments on this dataset and real-world samples demonstrate that our method can synthesize much better results than the state-of-the-art methods in both quantitative and qualitative comparisons.
Introduction and Problem Statement
In the rapidly evolving field of AI-driven fashion, precise attribute editing remains a significant challenge. The goal is to allow users to modify specific components of a garment—such as swapping a round collar for a V-neck or changing short sleeves to long sleeves—while maintaining the original fabric's texture and style.
Existing generative models, such as CycleGAN or Pix2pix, frequently struggle with this task due to two primary hurdles:
- Attribute Entanglement: In many neural architectures, the representation of texture (fabric pattern) and structure (garment shape) are deeply intertwined. Attempting to modify the structure often results in unintended distortions of the texture.
- Data Scarcity: Supervised learning typically requires "paired" data—images of the exact same garment with different attributes. Such datasets are virtually non-existent in the fashion domain.
TailorGAN addresses these issues by proposing a disentangled manipulation framework that operates without the need for paired training data.
Methodology
TailorGAN utilizes a two-stage training schema to decouple texture from geometry.
1. Self-Supervised Reconstruction
The first stage focuses on learning the relationship between local texture and global garment style. The model is presented with a garment image where a specific part (e.g., the collar) is masked out. It is then provided with the original edge map (a "sketch") of that part and tasked with reconstructing the missing area. This phase teaches the network how to map existing textures onto specific geometric coordinates.
2. Adversarial Learning for Attribute Manipulation
In the second stage, the model is challenged to perform actual edits. Instead of the original edge map, it receives an edge map representing a target attribute (e.g., a different collar type). An Attribute-Aware Discriminator (Attr-D) evaluates the output to ensure the generated part is both photo-realistic and matches the intended attribute category.
Key Technical Innovations
- Self-Attention Mask (SAM): Rather than generating a full image from scratch, the model outputs a mask and a set of new pixels. The SAM module then "pastes" the new pixels onto the original image, ensuring that only the target area is modified while the rest of the garment remains untouched.
- Geo-Transfer: To handle misalignments between the reference garment and the target attribute, the model uses random rotations and scaling of edge maps during training, making the synthesis robust to geometric variations.
Datasets: GarmentSet
To facilitate this research, the authors introduced GarmentSet, a benchmark dataset containing 9,636 images with collar annotations (12 types) and 8,616 images with sleeve/shoulder annotations. Unlike previous datasets like DeepFashion, GarmentSet features garments against clean, single-color backgrounds without human models. This reduction in environmental noise allows the model to focus purely on garment geometry and texture.
Experimental Results
TailorGAN was evaluated against state-of-the-art baselines using Classification Error (C.E.), Structural Similarity Index (SSIM), and Peak Signal-to-Noise Ratio (PSNR).
- Quantitative Performance: TailorGAN significantly outperformed Pix2pix and CycleGAN. In collar translation tasks (e.g., Type 1 to Type 2), TailorGAN achieved a C.E. of 9.17, compared to 20.01 for Pix2pix. This indicates that the generated attributes are much more recognizable as the intended target types.
- Qualitative Quality: Visual inspections confirm that TailorGAN produces sharper edges and maintains texture consistency far better than competing models, which often suffer from "texture bleed" or blurring.
- Zero-Shot Generalization: Notably, the model demonstrated the ability to synthesize collar types it had not encountered during the adversarial training stage, suggesting it learned a generalizable understanding of fashion geometry.
Conclusion
TailorGAN provides a robust solution for disentangled fashion attribute editing. By leveraging edge maps as a universal geometric language and employing a localized masking approach (SAM), it successfully bridges the gap between different garment styles. Future work may involve extending this framework to more complex structural modifications and integrating it into professional design tools or e-commerce platforms.
Summary updated