ReSFlow: Rectifying Categorical Flows on Statistical Manifolds for One-Step Generation

Xiaohuan Jia, Renzhe Xu, Xiao Wang, Jiayun Wu, Shaohua Fan

TL;DR

ReSFlow enables high-quality, one-step generation of discrete data (like text, images, and DNA) by "straightening" generative flows to follow the natural geodesics of statistical manifolds. By combining Riemannian geometry with an iterative "reflow" procedure, it achieves order-of-magnitude speedups over traditional multi-step flow models while maintaining or exceeding state-of-the-art sample quality.

What Problem Does This Paper Address?

Generative modeling for discrete data—think text tokens, DNA bases, or binary pixels—is notoriously tricky. Unlike continuous images, discrete variables don't have a natural differentiable structure. A common workaround is to map these variables onto a statistical manifold (specifically, the probability simplex), where we can treat categorical distributions as points in a curved space.

However, existing flow-based models on these manifolds suffer from a major efficiency bottleneck. Because they learn vector fields from independent pairings of noise and data, the resulting trajectories are often "tortuous" or highly curved. To generate a sample, you have to numerically integrate these curved paths using many small steps (often 50–100+), which is computationally expensive.

While "Rectified Flow" solved this for continuous data in Euclidean space by making paths straight lines, applying it to discrete data isn't straightforward. A straight line in flat Euclidean space might "fly off" the curved manifold, violating probability constraints or ignoring the intrinsic geometry of the data.

Methods

The authors propose Rectified Statistical Flow (ReSFlow), which brings the "straight and fast" property to curved manifolds. Their approach relies on two key pillars:

  1. Spherical Mapping: They use a square-root diffeomorphism to map the probability simplex onto the positive orthant of a hypersphere ($\mathbb{S}^{K-1}_+$). This is a brilliant move because it transforms the complex Fisher-Rao metric of the simplex into the standard, well-understood geometry of a sphere. On a sphere, the "straightest" paths (geodesics) are simply great-circle arcs, which can be calculated using Spherical Linear Interpolation (SLERP).
  2. Geodesic Reflow: They introduce an iterative "reflow" procedure. After training an initial model, they use it to generate deterministic pairings between noise and data. They then retrain the model to follow the geodesic (the SLERP path) between these new pairs. This process progressively "straightens" the learned trajectories.
Comparison of Euclidean vs. Spherical Reflow. Euclidean reflow (top) can still result in intersecting paths on the manifold, whereas ReSFlow (bottom) aligns paths with non-intersecting geodesics.

Once the flow is rectified, the paths are so close to true geodesics that the model can map noise to data in a single step using the Riemannian exponential map.

Datasets

The authors evaluated ReSFlow across diverse domains to prove its versatility:

  • Synthetic: A Swiss-roll distribution embedded on a 2-simplex.
  • Vision: Binarized MNIST (treating each pixel as a Bernoulli variable).
  • NLP: Character-level language modeling using the Text8 dataset (27 categories).
  • Bioinformatics: Conditional promoter DNA sequence design (4 categories: A, C, G, T).

Key Insights and Ideas

  • Geometry and Rectification are Synergistic: The "aha moment" is that you can't just straighten paths; you have to straighten them within the right geometry. Euclidean straightening on a manifold leads to geometric distortions that degrade performance.
  • Solving the "Boundary Illness": Operating directly on the simplex is numerically unstable because the Fisher information metric diverges at the boundaries (where probabilities are zero). Mapping to the hypersphere makes the math numerically stable everywhere.
  • Theoretical Guarantees: The paper proves that spherical reflow monotonically reduces the "transport cost" (the energy required to move from noise to data) and guarantees a straightening rate of $O(1/N_{reflow})$.

Key Results

ReSFlow's primary achievement is maintaining high quality while drastically reducing inference time.

  • Massive Speedup: On binarized MNIST, ReSFlow generates 1,000 samples in 0.45 seconds (one-step), compared to 112.8 seconds for the non-rectified SFM baseline—a 250x speedup.
  • One-Step Parity: In the MNIST task, ReSFlow's one-step generation (FID 5.97) is comparable to the baseline SFM requiring 34 steps (FID 5.92).
  • Superior Text Modeling: On Text8, ReSFlow achieved a Bits-Per-Character (BPC) of 1.26, significantly outperforming Euclidean-based rectified flows like ReDi (2.30) and non-rectified manifold flows (1.83).
Figure 2: Swiss-roll samples on the simplex under one-step Euler sampling.

Limitations and Future Work

The authors acknowledge a few key limitations:

  • Semantic Equidistance: The Fisher-Rao geometry assumes all categories are equally "different." In reality, some tokens (like "cat" and "dog") are semantically closer than others ("cat" and "apple"). The current model doesn't incorporate these semantic priors.
  • Discrete Tokenization: While ReSFlow works directly on raw categorical data, many SOTA models use pre-trained embeddings. Integrating ReSFlow with learned latent spaces could be a powerful next step.
  • Broader Manifolds: The authors suggest extending this framework to other non-Euclidean data, such as rotation-valued representations or directional statistics.

For more details, check out the full paper or the code at https://github.com/xiaohuanjia/ReSFlow.

Summary updated

Abstract

Generative modeling for discrete data has been significantly advanced by learning continuous vector fields on the statistical manifolds to transport simple priors to target discrete distributions. However, because these continuous-time flows are not constrained to follow geodesics on the statistical manifold, thereby requiring multi-step numerical integration for simulation and leading to substantial computational costs during inference. In this work, drawing inspiration from the "straight and fast" properties of Rectified Flow in continuous Euclidean space, we propose the Rectified Statistical Flow (ReSFlow) model, which enables one-step categorical generation on statistical manifolds by explicitly learning geodesic trajectories that adhere to the intrinsic Riemannian geometry. By rectifying the flow on the statistical manifold, we theoretically guarantee that trajectories between prior and target distributions approximate geodesics. This rectification significantly reduces the transport cost by aligning the learned trajectories directly along geodesics on the statistical manifold, allowing the model to map pure noise to target discrete data in a single step. Through extensive experiments on diverse real-world generative tasks, ranging from image and text to biological domains, we demonstrate ReSFlow's superior one-step generation quality, which truly embodies geodesically optimal and rapid categorical generation. Our code is available at https://github.com/xiaohuanjia/ReSFlow.
BibTeX
@misc{jia2026resflow,
  author = {Xiaohuan Jia and Renzhe Xu and Xiao Wang and Jiayun Wu and Shaohua Fan},
  title = {{ReSFlow: Rectifying Categorical Flows on Statistical Manifolds for One-Step Generation}},
  year = {2026},
  eprint = {2610.13225},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2610.13225},
}

Loading related content...

Figure