CURVTON-205K: A Mask-Free Difficulty-Stratified Synthetic Dataset for Robust and Scalable Virtual Try-On

Dipan Bartaula, Anuj Shrestha, Ankit Belbase, Anish Bishwakarma, Saroj Poudel, Samip Ghimire, Shashank Bhatta, Eric Arazo, Binod BhattaraiarXiv 2026 · arXiv

Virtual try-on (VTON) technology has come a long way, but it still often stumbles when faced with the messy reality of the "in-the-wild" world—think complex poses, layered clothing, and diverse cultural garments. A new paper introduces CURVTON-205K, a massive synthetic dataset designed to move the field beyond these bottlenecks by focusing on data diversity and controllable difficulty.

TL;DR

CURVTON-205K is a curated synthetic dataset of 205,000 virtual try-on triplets that eliminates the need for expensive manual annotations like segmentation masks. By using a Vision-Language Model (VLM) guided refinement loop, the authors created a high-quality, difficulty-stratified resource that significantly improves the robustness of try-on models in unconstrained real-world settings.

What Problem Does This Paper Address?

Most research in 2D virtual try-on focuses on building better neural network architectures. However, the authors argue that the real "failure modes"—like poor handling of occlusions (e.g., a hand over a shirt) or limited pose variety—are actually caused by the datasets we use.

Existing datasets like VITON-HD or DressCode are often small, studio-shot, and primarily feature Western-style clothing in simple frontal poses. Furthermore, they usually require "parsing masks" (pixel-perfect maps of where the person and clothes are), which are expensive to create and prone to errors. This makes current VTON systems "brittle": they work great in the lab but fail when a user uploads a selfie in a complex pose or wearing a traditional garment.

Methods

The researchers developed a scalable, multi-stage data generation pipeline that uses state-of-the-art AI models to "hallucinate" high-quality training data.

  1. Garment & Person Streams: Using LLMs and the FLUX.2 diffusion model, they generated thousands of diverse garment images and person portraits, covering a wide range of body types, ethnicities, and backgrounds.
  2. Difficulty-Aware Editing: They didn't just generate random images. They used specific editing commands to create "Easy," "Medium," and "Hard" versions of person images, introducing controlled levels of pose variation and occlusion.
  3. VLM-Guided Refinement: This is the "secret sauce." Instead of just throwing away bad generated images (rejection sampling), they used a Vision-Language Model (Qwen3-VL) as a critic. If the VLM detected an error (like a missing sleeve or a weird artifact), it provided feedback to the generator to fix the image in a refinement loop.
  4. Quality Screening: Finally, they trained a small ViT-based filter on a manually labeled subset to automatically score and prune the final dataset, ensuring only the most realistic triplets remained.
The multi-stage data generation pipeline including garment, person, and editing streams.

Datasets

The paper introduces CURVTON-205K, which consists of:

  • 205,000 training triplets (source person, target garment, and the resulting try-on).
  • 4,500 test triplets, manually verified for high quality.

The dataset is compared against and used to augment standard benchmarks:

  • VITON-HD & DressCode: The industry standards for studio-shot try-on.
  • StreetTryOn: A challenging "in-the-wild" dataset.
  • OpenVTON-Bench & WildVTON: Recent benchmarks focusing on diversity.

Key Insights and Ideas

  • Data-Centric Robustness: The study proves that many VTON failures aren't architectural flaws but "data gaps." By filling these gaps with synthetic data, existing models become significantly more robust without changing a single line of their code.
  • Refinement > Rejection: Using a VLM to provide constructive feedback to a generator is far more efficient than simple rejection sampling. It allows the pipeline to successfully create very difficult samples (like complex poses) that would otherwise be discarded.
  • Stratification Matters: By organizing the data into difficulty tiers, researchers can perform "curriculum learning"—starting training on easy samples and gradually moving to harder ones—which helps models converge better.
Examples of try-on results across Easy, Medium, and Hard difficulty tiers.

Key Results

The impact of CURVTON-205K is most apparent when testing on real-world, "in-the-wild" images:

  • Massive Gains in Realism: On the StreetTryOn benchmark, models trained with CURVTON-205K saw their FID (Fréchet Inception Distance, a measure of image quality) drop by 4.86 to 8.92 points, a substantial improvement in photorealism.
  • Cross-Dataset Boost: Even on standard benchmarks like VITON-HD, adding CURVTON-205K data to the training set improved performance across nearly all metrics (SSIM, LPIPS, FID, and KID).
  • Human Preference: In blind tests, human evaluators preferred the results from CURVTON-trained models 35% of the time for try-on quality, compared to just 10-15% for models trained on standard real-world datasets.

Limitations and Future Work

While the results are impressive, the authors acknowledge a few hurdles:

  • Synthetic-to-Real Gap: There is still a subtle difference between synthetic and real images that models must learn to bridge.
  • Fine-Grained Drift: In very complex "Hard" cases, tiny details like specific buttons or pocket shapes can sometimes be lost or slightly altered.
  • Demographic Auditing: While the dataset is diverse, the authors note it shouldn't be treated as a definitive demographic fairness assessment, and future work is needed to audit these aspects more deeply.

For those interested in exploring the data or the pipeline, the project page and code are available at https://curvton2.github.io/curvton/.

Summary updated

Abstract

Recent advances in 2D virtual try-on (VTON) have largely focused on architectural innovation, yet robust performance in diverse real-world conditions remains challenging. We argue that failure modes in VTON, such as inaccurate occlusion handling, unconstrained poses, and poor coverage of under-represented garments and appearance factors, are tightly coupled to dataset design choices and long-tail data regimes. We present a scalable data generation pipeline that coordinates specialized editing and generation models with vision-language model (VLM) guidance to synthesize triplets without requiring parsing masks or dense-pose annotations. Using this pipeline, we create CURVTON-205K, a curated synthetic dataset of 205K triplets: source person, target garment, and corresponding try-on image. CURVTON-205K is explicitly designed to broaden coverage across garments, person appearance, poses, occlusions, and visual conditions. Additionally, CURVTON-205K is organized into three difficulty tiers, enabling controlled evaluation under varying levels of difficulty and supporting difficulty-aware experimentation. We analyze the dataset diversity and demonstrate its potential to improve try-on fidelity across four VTON architectures and its cross-dataset generalization.
BibTeX
@misc{bartaula2026curvton,
  author = {Dipan Bartaula and Anuj Shrestha and Ankit Belbase and Anish Bishwakarma and Saroj Poudel and Samip Ghimire and Shashank Bhatta and Eric Arazo and Binod Bhattarai},
  title = {{CURVTON-205K: A Mask-Free Difficulty-Stratified Synthetic Dataset for Robust and Scalable Virtual Try-On}},
  year = {2026},
  eprint = {2611.24042},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.24042},
}

Loading related content...

Figure