One-to-Many: Diverse and Bidirectional Image-to-Image Translation via Rectified Flow

Sho Shimotsumagari, Seiichi Uchida, Shumpei TakezakiarXiv 2026 · arXiv

TL;DR

O2MFlow is a unified framework for image-to-image (I2I) translation that achieves both diverse "one-to-many" generation and exact "backward" translation (target-to-source) within a single model. By conditioning a Rectified Flow (RF) model on a learned appearance latent, the authors turn deterministic transport into a family of invertible bijections, allowing for high-fidelity, multimodal outputs and accurate source recovery.

What Problem Does This Paper Address?

Image-to-image translation is often inherently ambiguous. For example, a single edge map of a shoe could correspond to many valid photos with different colors, materials, or lighting. This is a one-to-many problem. However, most standard training datasets only provide one target image per source.

Existing frameworks typically fall into one of three camps, none of which are ideal:

  1. One-to-one, non-invertible: Models like pix2pix produce a single deterministic output and cannot be easily reversed.
  2. One-to-one, invertible: Standard Rectified Flow models are invertible but can only map one input to one output.
  3. One-to-many, non-invertible: Models like BicycleGAN produce diverse outputs but lack a mathematical inverse to map a target back to its source.

The challenge is to create a single model that can generate diverse, structure-preserving outputs while also being able to perfectly "un-translate" a target image back to its original source map.

Figure 1. Four representative settings of image-to-image transla- tion. The proposed method corresponds to (d), supporting one-to- many paired translation and backward translation within a single framework.

Methods

The authors propose One-to-Many Flow (O2MFlow), which builds upon Rectified Flow (RF). RF learns a velocity field that defines a straight-line ordinary differential equation (ODE) path between source and target distributions.

The key innovation is conditioning this velocity field $v(X_t, t, z)$ on an appearance latent $z$.

  • Training: An encoder $q(z|X_1)$ extracts a latent code from the target image. The model is trained to transport the source image to that specific target using the extracted latent.
  • KL Regularization: A Kullback–Leibler (KL) penalty is used to align the encoded latent distribution with a simple Gaussian prior $p(z) = \mathcal{N}(0, I)$. This ensures that at test time, we can simply sample new latents from the prior to generate diverse outputs.
  • Forward Translation: Sample $z \sim p(z)$ and integrate the ODE from $t=0$ to $t=1$.
  • Backward Translation: Given a target image, use the encoder to find its latent $z$, then integrate the same ODE backward from $t=1$ to $t=0$.
Figure 2. (a) Standard RF. (b) O2MFlow samples appearance latents for diverse forward translation and infers a latent from the target for backward translation. Here q denotes the target-only encoder and zX1 a latent inferred from target X1; parameter subscripts are omitted. The vertical axis represents image space.

Datasets

The model was evaluated on three standard paired image-to-image benchmarks:

  • edges2shoes: Sketch-to-photo of shoes.
  • edges2handbags: Sketch-to-photo of handbags.
  • facades: Semantic label-to-photo of architectural facades.

All images were resized to $64 \times 64$ for the main experiments, with some qualitative results shown at $128 \times 128$.

Key Insights and Ideas

  • Latent-Indexed Bijections: By fixing the latent $z$, the transport becomes a deterministic bijection. This is what allows for exact backward translation without needing a separate "backward" model or cycle-consistency losses.
  • Shared Latent Transport: The authors formalize the idea that an appearance latent learned from one image pair (e.g., "red leather") can be reused across different source images to produce consistent style variations while preserving the unique structure of each source.
  • The Role of KL Regularization: Beyond just encouraging diversity, the KL term mathematically controls the "distribution gap" between training (where latents come from targets) and generation (where latents are sampled). Minimizing this gap is crucial for the model to generalize to unseen sources.

Key Results

  • Superior Backward Translation: O2MFlow achieved the lowest backward LPIPS-Dist (a measure of how well the reconstructed source matches the original) across all three benchmarks. It also achieved the lowest backward FID on two out of three.
  • Diversity-Fidelity Trade-off: In forward translation, O2MFlow offers a competitive balance. It produces more diverse outputs (higher LPIPS-Div) than diffusion-bridge and standard flow baselines on the shoe and handbag datasets.
  • Numerical Reversibility: "Same-latent round trips" (forward then backward with the same $z$) showed extremely low error (LPIPS $\approx 0.001$ for shoes), proving the numerical stability of the invertible ODE framework.

Limitations and Future Work

  • Resolution: The quantitative evaluation was limited to $64 \times 64$ images. Scaling to higher resolutions remains a challenge for standard RF architectures.
  • Computational Cost: Like other ODE-based models, O2MFlow requires multiple network evaluations (NFE) per image during inference, which is slower than single-pass GANs.
  • Latent Structure: The current latents are unstructured. Future work could involve making these latents more interpretable or controllable (e.g., separate latents for color vs. texture).

Summary updated

Abstract

Many paired image-to-image translation tasks admit multiple valid target appearances, while training provides only one target per source. We propose One-to-Many Flow (O2MFlow), a latent-conditioned Rectified Flow (RF) model for one-to-many generation and backward translation (target$\rightarrow$source) within a single transport framework. Conditioning the RF dynamics on an appearance latent turns transport into a family of latent-indexed bijections: a fixed latent gives a mapping that is inverted by time-reversed integration, and sampled latents produce diverse outputs for the same input. We formalize reusable appearance variations as Shared Latent Transport and show that Kullback--Leibler (KL) regularization directly controls the source--latent distribution gap between training and generation. Across three benchmarks, O2MFlow offers dataset-dependent forward diversity--fidelity trade-offs and the lowest backward LPIPS-Dist among evaluated methods, with the lowest backward FID on two benchmarks.
BibTeX
@misc{shimotsumagari2026one,
  author = {Sho Shimotsumagari and Seiichi Uchida and Shumpei Takezaki},
  title = {{One-to-Many: Diverse and Bidirectional Image-to-Image Translation via Rectified Flow}},
  year = {2026},
  eprint = {2611.26058},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.26058},
}
Figure