FW-GAN: Flow-Navigated Warping GAN for Video Virtual Try-On
Introduction
Video virtual try-on represents a significant leap from static image-based systems, aiming to provide a realistic preview of how garments move and drape on a person in motion. While image-based methods have matured, extending them to video introduces severe challenges in spatiotemporal consistency. Naive frame-by-frame application often results in "texture flickering," where clothing patterns jump inconsistently between frames, failing to adhere to the body's natural movement.
The FW-GAN Framework
To address these issues, the Flow-navigated Warping GAN (FW-GAN) introduces a multi-stage architecture designed for temporal coherence and high-fidelity texture preservation. The core of the system is the Warping Net. Rather than generating clothing from scratch—which often blurs intricate details—the Warping Net learns a transformation grid to deform a flat image of the target garment to match the person's specific pose.
To maintain structural integrity, the model employs a Parsing Constraint Loss. This utilizes a human parser to ensure the synthesized body parts and clothing align with the expected anatomical layout for a given pose. This structural guidance is essential for handling complex movements and occlusions.
Furthermore, a Flow-Guided Fusion Module leverages optical flow to warp previous synthesized frames into the current one. This "temporal memory" ensures that pixels move logically across the sequence, significantly reducing flickering. The training is supervised by a dual-discriminator setup: a Frame Discriminator for per-frame realism and a Flow-embedding Discriminator to enforce physically plausible motion.
The VVT Dataset and Evaluation
The authors introduced the VVT (Video Virtual Try-on) Dataset to benchmark this task. Comprising 791 high-quality videos (approx. 190,000 frames) of fashion models, it offers a diverse range of viewpoints and complex movements compared to previous static datasets. Each video is paired with a specific person image and a target clothes image, enabling supervised training.
Quantitatively, FW-GAN achieves a Frechet Inception Distance (FID) of 7.05, a substantial improvement over the baseline CP-VTON's score of 32.35. Qualitative assessments and human studies further validate the approach, with users preferring FW-GAN's results in over 88% of comparisons due to its ability to maintain sharp logos and consistent textures during movement.
Conclusion and Future Directions
FW-GAN demonstrates that combining explicit warping with flow-guided temporal fusion can overcome the primary hurdles of video virtual try-on. While the system excels on clean backgrounds and standard tops, future work will focus on "in-the-wild" scenarios with complex backgrounds and a wider variety of garment types, such as layered outfits and full-body silhouettes.
Summary updated