MaX4Zero : Masked Extended Attention for Zero‐Shot Virtual Try‐On In The Wild

Nadav Orzech, Yotam Nitzan, Ulysse Mizrahi, Dov Danon, Amit H. BermanoComputer 2026 · DOI

TL;DR

MaX4Zero introduces a novel, training-free approach to virtual try-on (VTON) that leverages pre-trained diffusion models. By employing a two-stage process involving garment warping and masked extended attention, it achieves realistic garment transfer on unseen individuals and clothing items without requiring any fine-tuning, overcoming limitations of existing supervised methods.

What Problem Does This Paper Address?

Virtual Try-On (VTON) aims to realistically replace a garment in an image with a different one, preserving both the person's identity and the garment's characteristics. Current state-of-the-art methods typically rely on supervised learning, requiring extensive paired datasets of target images, reference garments, and desired outcomes. This approach is computationally expensive and suffers from poor generalization to unseen garments or individuals. The paper addresses the need for a zero-shot, training-free VTON method that can work "in the wild" on diverse images and garments, leveraging the inherent capabilities of powerful pre-trained generative models without costly fine-tuning.

Methods

MaX4Zero employs a two-stage pipeline:

  1. Initial Registration: This stage addresses the "texture sticking" problem, where reference garment features are treated as mere textures rather than semantic elements. It warps the reference garment onto the target person using deep features extracted from both images. This initial alignment ensures that garment features are positioned correctly. Any remaining gaps are filled using a "Fringe Assignment" module.
  2. Consistent Inpainting with Masked Extended Attention (MEA): This stage focuses on transferring fine-grained garment details and ensuring image fidelity. It utilizes a pre-trained diffusion model's prior. The core innovation here is Masked Extended Attention (MEA). Unlike standard self-attention, MEA expands the keys and values of the attention mechanism to incorporate information from the reference garment, while keeping queries fixed to the target person. This allows for semantic information transfer. Crucially, masking is applied to this extended attention to prevent leakage of the reference garment's background and unwanted influences into the target image. This mechanism is integrated into a stroke-based inpainting process (SDEdit) for fine detail enhancement.

Datasets

The paper uses publicly available datasets for evaluation:

  • Garment Images: DressCode, VITON-HD
  • Target Person Images: A large dataset for "in-the-wild" human parsing (likely referring to the dataset used in Liang et al., 2015).
  • Garment Descriptions: Extracted from Baldrati et al., 2023.

No new datasets or benchmarks were introduced.

Key Insights and Ideas

  • Leveraging Diffusion Model Priors: The paper argues that pre-trained diffusion models already possess the necessary knowledge for VTON and only require proper guidance, rather than task-specific fine-tuning.
  • "Texture Sticking" as a Key Challenge: The authors identify and address the problem of reference garment features being treated as textures, leading to unnatural results, through an initial warping step.
  • Masked Extended Attention for Controlled Transfer: The MEA mechanism is a novel way to inject reference garment information into the diffusion process. The masking component is critical for preventing background leakage and ensuring that only relevant garment details are transferred.
  • Zero-Shot Generalization: The training-free nature of the approach is key to its ability to generalize to unseen garments and individuals, a significant limitation of supervised methods.

Key Results

  • Qualitative: MaX4Zero generates realistic images with unchanged backgrounds and successfully preserves the target person's identity while accurately transferring garment texture and characteristics. It outperforms baseline methods in terms of garment preservation and realism, especially for unseen items.
  • Quantitative: In "in-the-wild" settings, MaX4Zero achieves competitive results, ranking second best in garment identity preservation and second best in person/background preservation, closely trailing methods that utilize inpainting. Its overall image fidelity (FID score) is comparable to state-of-the-art.
  • User Study: A user study with 57 participants indicated that MaX4Zero's results were preferred over benchmark methods across three aspects: target person identity/posture preservation, garment attribute capture, and overall image fidelity.

Limitations and Future Work

  • Limitations:
    • The model struggles to accurately reproduce readable and coherent textual elements (text or logos) on garments.
  • It is sensitive to mask placement; if the mask includes other body parts or extends beyond the garment, it can alter those elements or the overall image.
  • Future Work: The authors suggest exploring more semantic or latent approaches to mask handling to overcome sensitivity issues. They also encourage further exploration of tuning-free, zero-shot methods for image-based inpainting across various domains.

Summary updated

Abstract

Abstract Virtual try‐on is a highly active line of research, with increasing demand. Its task is replacing a piece of garment in an image with one from another while preserving person and garment characteristics as well as image fidelity. Current literature takes a supervised approach for the task, impairing generalisation and imposing heavy computation. In this paper, we present a novel zero‐shot training‐free method for embedding a clothing garment by reference. Our approach employs the prior of a diffusion model with no additional training, fully leveraging its native generalisation capabilities. The method employs extended attention to transfer image information from reference to target images, overcoming two significant challenges. We first initially warp the reference garment over the target human using deep features, alleviating ‘texture sticking.’ We then leverage the extended attention mechanism to encourage garment preservation while eliminating leakage of reference background and unwanted influence. Through a user study and a qualitative and quantitative comparison to state‐of‐the‐art approaches, we demonstrate superior image quality and garment preservation for unseen clothing pieces.
BibTeX
@inproceedings{orzech2026i,
  author = {Nadav Orzech and Yotam Nitzan and Ulysse Mizrahi and Dov Danon and Amit H. Bermano},
  title = {{<i>MaX4Zero</i> : Masked Extended Attention for Zero‐Shot Virtual Try‐On In The Wild}},
  booktitle = {Computer},
  year = {2026},
  doi = {10.1111/cgf.70303},
}
Figure