StegoGAN: Leveraging Steganography for Non-Bijective Image-to-Image Translation

Sidi Wu, Yizi Chen, Samuel Mermet, Lorenz Hurni, Konrad Schindler, Nicolas Gonthier, Loic Landrieu

Introduction

Standard image-to-image (I2I) translation frameworks, such as CycleGAN, typically operate under the assumption of bijectivity—the premise that every element in the source domain has a unique, direct counterpart in the target domain. However, real-world applications frequently involve non-bijective mappings. For instance, in medical imaging, a healthy scan should not be translated into one containing a tumor simply because the training set includes pathological examples. When faced with these "unmatchable" features, traditional GANs often "hallucinate" spurious details to satisfy the discriminator, compromising semantic integrity.

Methodology: Explicit Steganography

StegoGAN addresses this flaw by explicitly modeling the steganographic process in feature space. While standard models often "cheat" by hiding information in nearly invisible pixel patterns to satisfy cycle-consistency, StegoGAN harnesses this behavior as a feature to quarantine domain-specific information.

Matchability Disentanglement

The architecture incorporates a Matchability Disentanglement Module (M), a CNN that predicts a mask to split latent features into "matchable" and "unmatchable" components. During the backward cycle ($Target \to Source \to Target$), the model is forced to store unmatchable information (such as text on maps or tumors in MRI) in a dedicated feature-space path rather than the image itself.

The cycle-consistency mechanism in StegoGAN preserves unmatchable features through a hidden feature-space path, preventing them from being hallucinated in the source domain.

To prevent the network from reverting to covert pixel-level steganography, the authors inject low-amplitude Gaussian noise into the generated images. This perturbation destroys hidden pixel patterns, ensuring the model relies on the explicit feature-space disentanglement. At inference time, a "clean" translation is produced by simply ignoring the unmatchable feature path.

Key Insights and Results

A significant advantage of StegoGAN is its ability to perform unsupervised discovery of unmatchable features. The model learns to identify domain differences entirely on its own, with emergent masks highlighting specific objects like tumors or text without manual labeling.

An emergent unmatchability mask generated by the model, identifying domain-specific features without manual supervision.

Performance Benchmarks

The authors evaluated StegoGAN across several datasets, including a new benchmark, PlanIGN, consisting of aerial photos and topographic maps.

  • Hallucination Reduction: In brain MRI tasks (T1 to FLAIR), StegoGAN's false positive rate for tumors was 20x lower than CycleGAN.
  • Fidelity: On the PlanIGN dataset, StegoGAN achieved an RMSE of 22.5, significantly outperforming CycleGAN's 27.0.
  • Stability: The model maintained a near-zero false positive rate for unmatchable features even as their prevalence in the target domain increased, demonstrating superior robustness compared to state-of-the-art models like CUT and DRIT.

Conclusion

StegoGAN represents a shift in I2I translation by treating steganography as a tool for feature disentanglement rather than a bug to be suppressed. By isolating unmatchable features, the model ensures semantically accurate translations across non-bijective domains, making it particularly valuable for high-stakes applications in medicine and cartography.

Summary updated

Abstract

Most image-to-image translation models postulate that a unique correspondence exists between the semantic classes of the source and target domains. However this assumption does not always hold in real-world scenarios due to divergent distributions different class sets and asymmetrical information representation. As conventional GANs attempt to generate images that match the distribution of the target domain they may hallucinate spurious instances of classes absent from the source domain thereby diminishing the usefulness and reliability of translated images. CycleGAN-based methods are also known to hide the mismatched information in the generated images to bypass cycle consistency objectives a process known as steganography. In response to the challenge of non-bijective image translation we introduce StegoGAN a novel model that leverages steganography to prevent spurious features in generated images. Our approach enhances the semantic consistency of the translated images without requiring additional postprocessing or supervision. Our experimental evaluations demonstrate that StegoGAN outperforms existing GAN-based models across various non-bijective image-to-image translation tasks both qualitatively and quantitatively. Our code and pretrained models are accessible at https://github. com/sian-wusidi/StegoGAN.
BibTeX
@misc{wu2024stegogan,
  author = {Sidi Wu and Yizi Chen and Samuel Mermet and Lorenz Hurni and Konrad Schindler and Nicolas Gonthier and Loic Landrieu},
  title = {{StegoGAN: Leveraging Steganography for Non-Bijective Image-to-Image Translation}},
  year = {2024},
  eprint = {2403.20142},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2403.20142},
}
Figure