The Conditional Analogy GAN: Swapping Fashion Articles on People Images

Nikolay Jetchev, Urs BergmannWorkshops @ ICCV 2017 · DOI · arXiv

Introduction

In the fashion e-commerce industry, the high cost and slow pace of professional photography for every new arrival present a significant bottleneck. While "virtual try-on" technology is a long-sought solution, it typically relies on complex 3D modeling or rare paired datasets where the same person is photographed in multiple outfits. The Conditional Analogy GAN (CAGAN) addresses this by enabling unpaired image-to-image translation. It allows for "swapping" a clothing item on a human model using only a standalone product image as a reference, effectively treating the task as an image analogy.

The CAGAN Framework

CAGAN modifies specific fashion articles while preserving the person's identity and pose without requiring manually labeled segmentation data. The framework relies on several key technical components:

The Generator and Implicit Masking

Instead of synthesizing a full image, the Generator ($G$) outputs a 3-channel color image of the new garment and a 1-channel matting mask ($\alpha$). The final output is a blend: $\text{mask} \times \text{new\_clothes} + (1-\text{mask}) \times \text{original\_person}$. This approach forces the model to discover exactly where clothing ends and skin begins—a process of implicit segmentation—ensuring that faces and backgrounds remain untouched.

Adversarial Training and Negative Sampling

The Discriminator ($D$) is trained to distinguish not just "real" from "fake," but also whether a person is wearing the correct article. By using "negative examples" (real photos paired with product images of garments they are not wearing), the model learns the specific relationship between the person and the garment, rather than just general realism.

Cycle Consistency

Borrowing from CycleGAN principles, the model utilizes a cycle-consistency loss. This penalizes the network if swapping Shirt A for Shirt B and then swapping back to Shirt A does not reconstruct the original image. This constraint is vital for maintaining the person's identity throughout the transformation.

Cycle consistency ensures that swapping an article and then swapping it back preserves the original human image.

Experimental Results

The model was trained on a dataset from Zalando SE, featuring 15,000 images of human models and paired standalone upper-body garments.

Qualitative Success

CAGAN demonstrates remarkable geometric flexibility. It can handle transitions between different necklines (e.g., crew-neck to V-neck) and adjust the fit of various garments like hoodies while conforming to the model's pose and lighting. Most importantly, it generalizes well to unseen people and articles, proving it has learned a general "wearing" transformation.

Qualitative results showing the original garment ($y_i$), the person ($x_i$), the generated swap ($x_i^j$), and the target garment ($y_j$).

Limitations and Future Directions

Despite its success, CAGAN faces challenges with high-frequency details, such as intricate knit patterns or small logos. It also currently performs best against the clean, consistent backgrounds of studio photography. Future work aims to expand the model's scope to include full-body outfits and accessories, and to explore different color spaces like Lab to better separate texture from color information.

Summary updated

BibTeX
@inproceedings{jetchev2017conditional,
  author = {Nikolay Jetchev and Urs Bergmann},
  title = {{The Conditional Analogy GAN: Swapping Fashion Articles on People Images}},
  booktitle = {Workshops @ ICCV},
  year = {2017},
  doi = {10.1109/iccvw.2017.269},
}
Figure