Unified Architectures for Multi-Domain Image Translation

93 concepts66 papers1 proposal

In traditional image-to-image translation, mapping between two visual domains—such as turning a summer landscape into winter—typically requires a dedicated model for that specific pair. As the number of domains ($N$) increases, this approach fails to scale: creating a complete network of translations requires $N(N-1)$ separate generators.

Recent research has shifted toward unified multi-domain translation, where a single generator learns to map across any number of target domains. This paradigm replaces the quadratic complexity of pairwise models with a scalable, parameter-efficient architecture that leverages shared features across all tasks.

The Scalability of Star Topologies

The fundamental shift in this field was the move from pairwise mappings to a "star" topology. As Yunjey Choi et al. proposed in StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation, a single generator can be trained to translate an image to any target domain by taking a domain label as an auxiliary input.

This unified approach offers two primary advantages:

  1. Parameter Efficiency: Instead of storing dozens of models, a single network handles all transitions. Xing Cai et al. demonstrate in SingleGAN that weight sharing allows the model to learn general structural information more effectively than isolated pairs.
  2. Cross-Dataset Training: Unified models can learn from multiple datasets with different label sets simultaneously. For instance, a model can learn "hair color" from one dataset and "facial expression" from another, enabling it to perform both transformations on a single image even if no training data exists for that specific combination.

Mechanisms of Domain Conditioning

To make a single generator versatile, the model must be "told" which domain to target. Researchers have developed several mechanisms to inject this information:

  • Label Vectors and Masks: Dongwook Lee et al. in CollaGAN use spatial masks or one-hot vectors to specify missing data types in medical imaging. Similarly, Ming Liu et al. utilize attribute difference vectors in STGAN to selectively edit features like age or gender without degrading unrelated image areas.
  • Normalization Layers: A common technique involves injecting domain information through normalization. Andrés Romero et al. show in SMIT that using Adaptive Instance Normalization (AdaIN) to inject domain embeddings allows the model to scale to numerous labels without increasing the parameter count.
  • Hypernetworks: For even greater flexibility, the authors of Hyper-GAN use a secondary "hyper-network" to predict the weights of a shared encoder-decoder based on the desired contrast. This allows for high-fidelity translation between various MRI modalities using a fraction of the parameters required by specialized models.

Learning Shared Representations

A core intuition behind unified models is that different domains often share underlying structures. In medical imaging, for example, a CT scan and an MRI of the same patient share the same anatomy, differing only in "style" or contrast.

Kai Hu et al. argue in One-to-Multiple that training a single generator to adapt a source domain to multiple targets simultaneously captures these shared anatomical features. This acts as a form of regularization; by learning to synthesize multiple styles (e.g., different histological stains as shown by Yiyang Lin et al. in Unpaired Multi-Domain Stain Transfer), the model develops a more robust understanding of the content it is translating.

Deterministic Bridges and Diffusion

While GANs have dominated this space, new methods use the deterministic properties of diffusion models to bridge domains. Xuan Su et al. introduced Dual Diffusion Implicit Bridges (DDIBs), which treat translation as a two-step process: encoding an image into a shared latent space using one domain's diffusion model, and then decoding it using another's.

This approach represents the logical extreme of decoupling: it requires no joint training between domains at all. Each domain model is trained independently, yet they can be "bridged" at inference time, allowing the system to scale linearly as new domains are added while maintaining cycle consistency.

Efficiency in Deployment

The move toward unified models is also driven by practical constraints. In person re-identification across large camera networks, training separate models for every camera pair is computationally prohibitive. Chuan-Xian Ren et al. demonstrate that a single many-to-many generative architecture is more efficient at capturing complex environmental variations like lighting and background noise across multiple views (Domain Adaptive Person Re-Identification).

By consolidating multiple tasks into a single framework, researchers have achieved significant reductions in memory and latency—sometimes slashing parameters by over 70%—making these powerful generative tools viable for real-world clinical and surveillance applications.

Key Foundational Work

StarGAN: Unified Generative Adversarial Networks for Multi-Domain Image-to-Image Translation (2017) — contributed 2 concept elements to this direction.
A Multi-Domain Translation Model Framework for Statistical Machine Translation (2013) — contributed 1 concept elements to this direction.
History & citationVersion 2 · 2026-10-04

Current · Permanent link

  1. Version 2 (latest) · 2026-10-04 · 70 papers
  2. Version 1 · 2026-10-04 · 94 papers
Cite this direction

Latest version (a living page):

@misc{lacuna_unified_multi_domain_translation_f5df10df,
  title = {unified multi-domain translation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/unified-multi-domain-translation/rdr_f5df10df673645e8bc5a57df2f4282e6},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 2, frozen text):

@misc{lacuna_unified_multi_domain_translation_f5df10df_v2,
  title = {unified multi-domain translation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/unified-multi-domain-translation/rdr_f5df10df673645e8bc5a57df2f4282e6/v/2},
  note = {Version 2, frozen 2026-10-04. Accessed 2026-11-19},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake