Style-Guided and Disentangled Representation for Robust Image-to-Image Translation

Jaewoong Choi, Dae Ha Kim, Byung Cheol SongAAAI 2022 · DOI

Introduction

Image-to-image (I2I) translation is a fundamental task in computer vision, aiming to map images from a source domain to a target domain while preserving structural content. While Generative Adversarial Networks (GANs) have enabled significant progress, they frequently suffer from "mode collapse," where the generator produces a limited variety of outputs, failing to capture the full diversity of the target domain. Researchers from Inha University have proposed SRIT (Style-Guided and Disentangled Representation), a framework designed to overcome these limitations by introducing flexible discrimination boundaries and information-theoretic regularization.

The Problem: Static Decision Boundaries

The authors identify the static decision boundary of traditional discriminators as a primary cause of mode collapse. In standard GANs, the discriminator maintains a fixed definition of the target domain. Consequently, the generator often converges to a "safe" domain average—a single basin in the probability distribution—rather than exploring the diverse styles (e.g., different breeds, lighting, or textures) present in real-world data.

The SRIT Architecture

To address these issues, SRIT introduces three core components:

  1. Style-Guided Discriminator (STGD): Unlike standard discriminators that evaluate images in isolation, STGD uses pairwise inputs. By comparing a generated image against a specific real reference image, the discriminator creates a flexible, relativistic decision boundary. This forces the generator to match the specific style of the reference rather than a generic domain mean.
  2. Normalized Point-Wise Mutual Information (NPMI): To ensure that style representations are distinct and independent, the authors apply NPMI as an extrinsic regularization term. This prevents "style bleeding" and encourages the latent space to disentangle various domain attributes.
  3. Importance Weighting Generator (IG) Loss: Drawing from information theory, this weighting scheme prioritizes parameter updates that produce the most realistic images, effectively boosting performance during the final training stages.
The SRIT framework consists of a style encoder (a), mapping network (b), generator (c), and a relativistic discriminator (d).

Experimental Evaluation

The SRIT model was rigorously tested on high-resolution benchmarks, including CelebA-HQ (faces), AFHQ (animals), and Yosemite (landscapes). The results demonstrate a significant leap over previous state-of-the-art (SOTA) models like StarGAN-v2 and MSGAN.

Quantitatively, SRIT achieved a Frechet Inception Distance (FID) of 12.4 on CelebA-HQ and 12.5 on AFHQ, representing a 22.8% improvement over StarGAN-v2 on the animal dataset. Furthermore, higher LPIPS scores confirm that SRIT generates more diverse images, successfully mitigating mode collapse.

Qualitative comparison across datasets and a discrimination plot showing SRIT's proximity to the equilibrium line compared to SOTA models.

As shown in the discrimination plot above, SRIT consistently stays closer to the equilibrium line ($D(x_r) = D(x_g)$) than its predecessors. This visualizes the model's ability to reach a global optimum where the discriminator can no longer easily distinguish between real and generated styles, leading to better visual quality.

Conclusion

SRIT represents a significant advancement in robust I2I translation. By replacing rigid discriminator boundaries with a style-guided approach and leveraging information theory for representation disentanglement, the framework achieves superior visual quality and diversity. While future work may address the computational overhead of the boosting phase and sparse "edge case" data, SRIT sets a new benchmark for high-fidelity, multi-domain image synthesis.

Summary updated

BibTeX
@inproceedings{choi2022style,
  author = {Jaewoong Choi and Dae Ha Kim and Byung Cheol Song},
  title = {{Style-Guided and Disentangled Representation for Robust Image-to-Image Translation}},
  booktitle = {AAAI Conference on Artificial Intelligence},
  year = {2022},
  doi = {10.1609/aaai.v36i1.19924},
}
Figure