Abstract
Attribute editing has become an important and emerging topic of computer vision. In this paper, we consider a task: given a reference garment image A and another image B with target attribute (collar/sleeve), generate a photorealistic image which combines the texture from reference A and the new attribute from reference B. The highly convoluted attributes and the lack of paired data are the main challenges to the task. To overcome those limitations, we propose a novel self-supervised model to synthesize garment images with disentangled attributes (e.g., collar and sleeves) without paired data. Our method consists of a reconstruction learning step and an adversarial learning step. The model learns texture and location information through reconstruction learning. And, the model's capability is generalized to achieve single-attribute manipulation by adversarial learning. Meanwhile, we compose a new dataset, named GarmentSet, with annotation of landmarks of collars and sleeves on clean garment images. Extensive experiments on this dataset and real-world samples demonstrate that our method can synthesize much better results than the state-of-the-art methods in both quantitative and qualitative comparisons.
Introduction
In the world of AI-driven fashion, "attribute editing" is a significant challenge. Imagine taking a floral print shirt and instantly seeing how it would look with a Peter Pan collar versus a lapel. TailorGAN is a novel generative framework designed to solve this by allowing users to swap specific fashion attributes—like changing a round collar to a V-neck or short sleeves to long sleeves—while seamlessly preserving the garment's original texture.
The primary technical hurdle is entanglement, where texture (pattern) and structure (shape) are deeply intertwined in generative models. Standard tools like CycleGAN often inadvertently change the color of the entire garment or blur the fabric pattern when attempting to modify a single attribute. Furthermore, training these models typically requires "paired data"—images of the exact same garment with only one attribute changed—which is virtually non-existent in real-world datasets.
Methodology
TailorGAN addresses these challenges through a specialized architecture and a two-stage training schema that decouples geometry from rendering.
1. Self-Supervised Reconstruction
The model first learns to "repair" a garment. It receives an image where the target attribute (e.g., the collar) is masked out, along with an edge map (a sketch) of the original attribute. By learning to fill this area with the correct original texture, the model establishes a fundamental understanding of the relationship between a garment's overall style and its structural components.
2. Adversarial Learning and Attribute Control
Once the model masters reconstruction, it is tasked with generating a new attribute using an edge map from a different source garment. An attribute-aware discriminator ensures the result is photorealistic and matches the target attribute type (e.g., ensuring a V-neck edge map results in a V-neck image). To maintain the integrity of the rest of the garment, a Self-Attention Mask (SAM) layer identifies exactly which pixels to modify and which to leave untouched.
3. Disentanglement via Edge Maps
Instead of raw RGB images, TailorGAN uses gray-scale edge maps (generated via Holistically-nested Edge Detection) as conditional inputs. This forces the generator to ignore the color and texture of the source attribute, focusing solely on its geometric shape.
Datasets and Evaluation
The researchers introduced GarmentSet, a new benchmark containing 9,636 images with collar annotations and 8,616 with sleeve annotations. Unlike previous datasets, GarmentSet features garments on clean, single-color backgrounds, allowing the model to focus purely on fashion design without background noise.
In comparative tests, TailorGAN significantly outperformed baselines like CycleGAN and Pix2pix:
- Quantitative Results: In collar translation tasks, TailorGAN achieved a Classification Error (C.E.) of 9.17, compared to 20.01 for Pix2pix, while maintaining higher Structural Similarity (SSIM) scores.
- Qualitative Results: Visual inspections show that TailorGAN preserves complex textures, such as stripes and logos, across the edit boundary, whereas other models often produced blurry artifacts or "color bulks."
Discussion and Future Work
A key insight of this work is that reconstruction serves as a vital "warm-up" for the latent space; if a model cannot reconstruct an original attribute, it cannot reliably synthesize a new one. By constraining the model with the SAM layer, the authors prevented the "hallucination" of changes in unrelated parts of the garment.
However, limitations remain. The final image quality is heavily dependent on the clarity of the input edge maps. Additionally, complex transitions—such as extending short sleeves to long sleeves—require the model to "invent" significant amounts of new texture, which can occasionally lead to artifacts. Future iterations could see TailorGAN integrated into professional fashion design assistants or e-commerce systems, allowing users to customize designs before purchase.
Summary updated