1-2-1: Renaissance of Single-Network Paradigm for Virtual Try-On

Shuliang Ning, Yipeng Qin, Xiaoguang HanarXiv 2025 · DOI · arXiv

Abstract

Virtual Try-On (VTON) has recently trended toward complex "dual-network" architectures to preserve garment details, but at a heavy cost to computational efficiency. This paper introduces MN-VTON, a single-network approach that utilizes a novel "Modality-specific Normalization" strategy to outperform state-of-the-art dual-network models in both image and video try-on tasks while significantly reducing overhead.

Introduction

Virtual Try-On is the task of realistically "overlaying" a garment onto a person in an image or video. Historically, the industry has navigated a tug-of-war between two paradigms:

  1. Single-Network: Early models used one generative network. They were efficient but often "washed away" fine-grained textures and logos because the network struggled to balance person-features and garment-features simultaneously.
  2. Dual-Network: To fix this, recent state-of-the-art (SOTA) methods added a second "ReferenceNet" specifically to extract garment details. While effective at preserving textures, this doubles the computational load, making high-resolution video try-on prohibitively expensive for real-world deployment.

The authors propose a "renaissance" of the single-network paradigm (the "1-2-1" in the title refers to this shift: 1 network $\rightarrow$ 2 networks $\rightarrow$ back to 1 network). They aim to prove that a single network can handle complex multi-modal inputs (text, garment images, and person videos) if the data is processed correctly.

Evolution of VTON paradigms (a, b, c) and qualitative results of MN-VTON across image and video benchmarks.

Methodology: MN-VTON

The core innovation is MN-VTON (Modality-specific Normalization Virtual Try-On). Instead of using a separate network to help the main model "remember" what the clothes look like, the intelligence is baked directly into the main network using two key strategies:

Modality-specific Normalization (MN)

The network receives three types of input: Text (descriptions), Garment Images, and Target Images/Videos. The authors found that treating these differently is key. They apply separate normalization layers (specifically AdaLN-zero) to the text features versus the visual features (garment + person). This prevents the "semantic" text data from interfering with the "structural" visual data.

Feature Split and Fusion

Within the attention layers of the network, internal features are split to mimic the behavior of a dual-network system. This allows the model to learn multi-scale garment features and fuse them with the person's pose and identity without needing an entirely separate set of weights.

Diffusion Transformer Backbone

Unlike many VTON models that use a standard UNet, MN-VTON is built on a Diffusion Transformer (DiT) architecture (specifically CogVideo-2B). This makes it naturally better at handling temporal consistency for video tasks.

The MN-VTON architecture featuring Modality-specific Normalization and DiT blocks.

Experimental Setup

The model was evaluated on four major benchmarks:

  • Image VTON: VITONHD (high-resolution top-wear) and DressCode (diverse upper, lower, and full-body dresses).
  • Video VTON: VVT (standard benchmark for moving subjects) and VIVID (high-quality video dataset for fine-grained detail and temporal stability).

Key Insights and Results

A major finding is that the Garment Image and the Target Person Image share similar visual distributions. By concatenating and normalizing them together—while keeping the Text modality separate—the network produces much clearer, artifact-free results compared to treating all three independently.

Performance and Efficiency

MN-VTON sets new benchmarks across the board:

  • Image Quality: On VITONHD, it achieved an SSIM of 0.8853, outperforming the previous SOTA, CatVTON (0.8704).
  • Video Fidelity: In video tasks (VIVID dataset), it achieved a VFID (Video Frechet Inception Distance) of 0.9237, a significant improvement over the baseline VIVID model's 1.1394.
  • Efficiency: By eliminating the "ReferenceNet," the authors reduced the number of trainable parameters significantly (e.g., from ~1700M in some dual-network setups to ~860M for their UNet-based variant) while actually improving accuracy metrics.
  • Human Preference: In user studies, participants preferred MN-VTON's results 65-68% of the time over competing methods, citing better preservation of logos and smoother motion in videos.

The model also demonstrates "mask-free" potential, generating realistic results even when person masks are missing or inaccurate, a common failure point for traditional models.

Conclusion

While MN-VTON is more efficient than dual-network models, it still relies on large-scale Diffusion Transformers requiring significant GPU resources. Future work may address extreme poses and scaling for 4K resolution. Overall, MN-VTON proves that a well-designed single-network architecture can reclaim the efficiency of early models without sacrificing the high-fidelity detail of modern dual-network systems.

Summary updated

Abstract

Virtual Try-On (VTON) has become a crucial tool in ecommerce, enabling the realistic simulation of garments on individuals while preserving their original appearance and pose. Early VTON methods relied on single generative networks, but challenges remain in preserving fine-grained garment details due to limitations in feature extraction and fusion. To address these issues, recent approaches have adopted a dual-network paradigm, incorporating a complementary "ReferenceNet" to enhance garment feature extraction and fusion. While effective, this dual-network approach introduces significant computational overhead, limiting its scalability for high-resolution and long-duration image/video VTON applications. In this paper, we challenge the dual-network paradigm by proposing a novel single-network VTON method that overcomes the limitations of existing techniques. Our method, namely MNVTON, introduces a Modality-specific Normalization strategy that separately processes text, image and video inputs, enabling them to share the same attention layers in a VTON network. Extensive experimental results demonstrate the effectiveness of our approach, showing that it consistently achieves higher-quality, more detailed results for both image and video VTON tasks. Our results suggest that the single-network paradigm can rival the performance of dualnetwork approaches, offering a more efficient alternative for high-quality, scalable VTON applications.
BibTeX
@misc{ning2025renaissance,
  author = {Shuliang Ning and Yipeng Qin and Xiaoguang Han},
  title = {{1-2-1: Renaissance of Single-Network Paradigm for Virtual Try-On}},
  year = {2025},
  eprint = {2501.05369},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2501.05369},
}
Figure