Unpaired Image-to-Image Translation with Shortest Path Regularization

Shaoan Xie, Yanwu Xu, Mingming Gong, Kun Zhang

Abstract

Unpaired image-to-image (I2I) translation remains a challenging task due to its ill-posed nature. Traditional methods like CycleGAN and contrastive learning (CUT) rely on cycle consistency or patch-wise mutual information, which can be overly restrictive or sensitive to sampling. This paper introduces Shortest Path Regularization, a novel approach that treats image domains as points on a continuous manifold. By assuming that the optimal translation follows the shortest path through a shared latent space, the proposed method achieves superior semantic preservation and image quality. Experimental results across multiple benchmarks, including Cityscapes and AFHQ, demonstrate that this approach outperforms state-of-the-art baselines in both visual fidelity and computational efficiency.

Introduction

Unpaired Image-to-Image (I2I) translation is a classic problem in computer vision: given two sets of images from different domains (e.g., horses and zebras) without direct correspondences, how can we learn a mapping that transforms an image from one domain to the other while preserving its underlying content?

Existing solutions usually rely on heavy-handed assumptions. CycleGAN assumes "cycle consistency"—the idea that translating an image to a target domain and back should yield the original image. However, this can be too restrictive, forcing the network to hide information in high-frequency noise. Contrastive learning (CUT) assumes that patches of the image should maintain mutual information, but this method is highly sensitive to the selection of "negative" samples. The authors of this paper argue that these methods treat domains as discrete, isolated islands, ignoring the continuous structure of the image manifold.

The Shortest Path Assumption

The core innovation of this work is the Shortest Path Assumption. The authors posit that for any image $x$, its "true" counterpart in another domain should be the one reachable via the shortest possible path on the image manifold. If the path is long, it suggests the network is adding unnecessary changes or "hallucinating" features that were not present in the source image.

Conceptual illustration of the shortest path assumption: the model seeks the most direct route between domains to preserve content.

Methodology

To implement this concept, the researchers developed a framework centered on three pillars:

  1. Shared Latent Space: An encoder $E$ and decoder $G$ are shared across both domains, ensuring a common representational foundation.
  2. Continuous Domain Variable ($\theta$): Instead of treating domains as binary states, they introduce a variable $\theta$ ranging from 0 (Source Domain) to 1 (Target Domain). This allows the model to generate intermediate images along a continuous path.
  3. Path Length Regularization ($L_{path}$): The model minimizes the length of the path generated as $\theta$ moves from 0 to 1. To avoid the computational expense of calculating a full Jacobian matrix, the authors use a central finite difference approximation to estimate the path's velocity and penalize its squared norm.

This shortest-path constraint is applied not just to the final pixels, but also to intermediate feature layers of the decoder. This ensures that semantic consistency is maintained throughout the entire generation process, from abstract concepts to fine-grained textures.

Key Results

The method, dubbed "Ours" in the paper, was evaluated on standard benchmarks including Cityscapes, AFHQ (cat $\to$ dog), and Horse $\to$ Zebra. It demonstrated significant improvements across several metrics:

  • Semantic Accuracy: On the Cityscapes task (label $\to$ city), it achieved a 31.0 mAP, outperforming the previous best of 29.0 (SRC) and 24.7 (CUT).
  • Image Quality: On the Cat $\to$ Dog task, it achieved a Fréchet Inception Distance (FID) of 52.1, a substantial improvement over the 76.2 achieved by CUT.
  • Efficiency: The approach is significantly faster than its competitors. It clocked in at 0.136 seconds per iteration, making it faster than CycleGAN (0.171s) and nearly twice as fast as NEGCUT (0.275s).
Qualitative comparison showing that the shortest path method avoids artifacts and preserves semantic structure better than baselines.

Qualitatively, the model avoids common artifacts, such as "zebra stripes" appearing in the sky or grass, which often plague contrastive methods. Because the model learns a continuous path, it can also generate smooth interpolations between domains (e.g., a "half-cat, half-dog"), which serve as a validation of the learned manifold's logical structure.

Conclusion

By shifting the focus from discrete domain mapping to continuous path optimization, Shortest Path Regularization provides a more principled and efficient way to handle unpaired I2I translation. The "minimalist" approach of the shortest path naturally preserves content identity while allowing for high-fidelity style transformation. Future work may explore more sophisticated path length approximations and extend the framework to handle multi-domain translation simultaneously.

Summary updated

Abstract

Unpaired image-to-image translation aims to learn proper mappings that can map images from one domain to another domain while preserving the content of the input image. However, with large enough capacities, the network can learn to map the inputs to any random permutation of images in another domain. Existing methods treat two domains as discrete and propose different assumptions to address this problem. In this paper, we start from a different perspective and consider the paths connecting the two domains. We assume that the optimal path length between the input and output image should be the shortest among all possible paths. Based on this assumption, we propose a new method to allow generating images along the path and present a simple way to encourage the network to find the shortest path without pair information. Extensive experiments on various tasks demonstrate the superiority of our approach. The code is available at https://github.com/Mid-Push/santa.
BibTeX
@inproceedings{xie2023unpaired,
  author = {Shaoan Xie and Yanwu Xu and Mingming Gong and Kun Zhang},
  title = {{Unpaired Image-to-Image Translation with Shortest Path Regularization}},
  booktitle = {IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year = {2023},
  doi = {10.1109/cvpr52729.2023.00981},
}
Figure