Generative Style Synthesis for Domain Adaptation

14 concepts13 papers1 proposal

Machine learning models often fail when deployed in environments that differ from their training data. A model trained on sunny daytime photos struggles in the snow; a detector trained on natural images fails to recognize objects in a painting. This "domain gap" typically requires expensive manual labeling of new data for every possible environment.

Generative style synthesis offers a different path: instead of labeling new data, we can use generative models to "re-skin" existing labeled data. By translating the visual style of a source domain into the characteristics of a target domain, researchers can create synthetic, labeled training sets that bridge the gap between disparate environments.

Bridging the Gap via Style Transfer

The core intuition behind this research direction is that the content of an image (e.g., the presence of a car) can be decoupled from its style (e.g., lighting, weather, or artistic medium). If we have a labeled dataset of photos, we can use style transfer to generate a "pseudo-target" dataset.

Kovashka et al. demonstrated this by using style transfer to recognize objects in art without any labeled artistic data. By forcing a network to learn from synthetic images that mimic paintings or sketches, the model develops features invariant to artistic style. Similarly, Kadish et al. introduced StyleCOCO, a large-scale dataset created by applying artistic styles to the standard COCO dataset, significantly improving object detection in art images.

This approach is not limited to vision. In natural language processing, Mukherjee et al. applied a similar logic to chatbots, using a fine-tuned BART model to perform "politeness transfer." By synthesizing polite dialogue from unlabeled text, the authors bypassed the need for manual annotation of polite conversation pairs.

Adapting with Minimal Data

In many real-world scenarios, we might have access to a few unlabeled images from the target domain—such as a handful of photos from a specific city or a new weather condition. Generative models can leverage these few samples to guide the synthesis process.

Berton et al. showed that as few as five unlabeled images are sufficient to adapt a geolocalization model to a new city by translating source images into the target's specific environmental style. For object detection, Meng et al. proposed "re-skinning" an entire labeled source dataset into a target style using few-shot image-to-image translation. This allows a standard detector to be trained as if it had a full labeled dataset in the new environment.

For more complex tasks like semantic segmentation, Wang et al. used an image translation engine to generate pseudo-target images that preserve source labels while adopting target textures. These synthetic images serve as anchors, "pulling" the model's understanding of the target domain closer to the well-understood source domain.

Hallucinating Unseen Domains

While domain adaptation focuses on a known target, Domain Generalization (DG) aims to prepare models for environments that are completely unknown during training. As surveyed by Zhou et al., generative models play a crucial role here by artificially expanding the diversity of the training set.

Rather than translating to a specific target, researchers use Generative Adversarial Networks (GANs) to synthesize a wide variety of "fake" domains. Rahman et al. used GANs to translate source images into multiple diverse styles, treating these synthetic variations as proxies for the unknown target.

Taking this a step further, Zhou et al. proposed a framework called DDAIG, which uses a transformation network to "hallucinate" adversarial domains. By training on these challenging, synthetic perturbations, the model learns to ignore domain-specific characteristics and focus on the underlying semantic content. This generative augmentation ensures that when the model encounters a truly new environment, it has already seen enough synthetic variation to remain robust.

Key Foundational Work

Efficient Multi-Domain Dictionary Learning with GANs (2019) — contributed 2 concept elements to this direction.
Artistic Object Recognition by Unsupervised Style Adaptation (2018) — contributed 1 concept elements to this direction.
Multi-component Image Translation for Deep Domain Generalization (2018) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_generative_domain_augmentation_b4b10e1f,
  title = {generative domain augmentation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/generative-domain-augmentation/rdr_b4b10e1f5f2649f084029556c6daf902},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_generative_domain_augmentation_b4b10e1f_v1,
  title = {generative domain augmentation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/generative-domain-augmentation/rdr_b4b10e1f5f2649f084029556c6daf902/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake