Correcting Flows with Marginal Matching

Meital Bojan, Nadav Bojan Sellam, Ev Zisselman, Aviv TamararXiv 2026 · arXiv

TL;DR

This paper introduces Marginal Matching (MM), a novel training-based predictor-corrector framework designed specifically for flow matching models. By exploiting the reversibility of ordinary differential equations (ODEs), the method steers generative trajectories toward a "perfect" backward path derived from real data, significantly improving image quality (FID) on benchmarks like CIFAR-10 and ImageNet-64 with minimal computational overhead.

What Problem Does This Paper Address?

Flow matching (FM) models have gained popularity as an efficient alternative to diffusion models. They work by learning a vector field that "flows" a simple source distribution (like Gaussian noise) into a complex target distribution (like natural images) via an ODE.

However, there is a catch: neural networks are imperfect. Even if you use a perfect ODE solver, the learned flow field won't map noise to perfect images because of training errors and model capacity limits. In diffusion models, "predictor-corrector" schemes (like Langevin dynamics) are used to "nudge" samples back toward the data manifold during generation. For flow matching, such schemes are less established. The authors aim to close this gap by providing a principled way to correct the trajectory of a flow model during inference.

Methods

The core innovation is Marginal Matching. The authors observe that if you take real data and flow it backward in time using the learned model, you get a trajectory that is "perfect" for that specific model. If the model were perfect, the forward path (starting from noise) and this backward path (starting from data) would be identical. In practice, they diverge.

Illustration of the gap between forward and backward marginals. Note that the backward path starting from data reaches a different "noise" point than the standard Gaussian.

The MM framework alternates between:

  1. Predictor Step: A standard ODE solver step (e.g., Euler or RK4) that moves the sample forward in time.
  2. Corrector Step: A learned function that "nudges" the sample from the current forward marginal distribution toward the corresponding backward marginal distribution.

The authors propose two ways to implement the corrector:

  • Score Model: An auxiliary model trained to estimate the score (gradient of the log-density) of the backward marginals.
  • Robust Classifier: A time-dependent classifier trained to distinguish between forward and backward samples. The gradient of this classifier provides the direction needed to steer samples toward the backward path.
The objective is to transition from the lower forward trajectory to the upper backward trajectory that accurately reaches the data distribution.

Datasets

The authors evaluated their method on two standard image generation benchmarks:

  • CIFAR-10: Unconditional image generation at 32x32 resolution.
  • ImageNet-64: A more complex dataset of 64x64 images.

Key Insights and Ideas

  • Exploiting Reversibility: Unlike diffusion models, which are stochastic, flow matching is deterministic. This makes it much easier to compute the "ideal" backward trajectory from training data.
  • Theoretical Grounding: The authors provide a proof showing that their correction scheme reduces a bound on the Wasserstein distance between the generated samples and the true data distribution.
  • Correction Prioritization: Not all time steps are equal. The theory suggests that for certain error types, corrections at earlier stages of generation have a larger impact on the final result.
  • Efficiency: The auxiliary correction models are smaller (roughly half the parameters) than the base flow model and require significantly fewer training iterations to converge.
Ablation study showing how FID improves as more correction steps are added at different time intervals.

Key Results

The Marginal Matching approach consistently outperformed baseline flow matching models with very little extra compute:

  • CIFAR-10: The base OT-CFM model achieved an FID of 4.34 with 40 function evaluations (NFE). Adding just one score correction step improved this to 3.45 (41 NFE). With 11 correction steps, the FID dropped to 3.37.
  • ImageNet-64: The base model's FID of 17.99 (at 20 NFE) was slashed to 15.53 with just two correction steps (22 NFE).
  • Qualitative Improvement: Visual results show that the corrector effectively removes residual noise and sharpens details that the base flow model misses.
Qualitative comparison showing how correction models improve sharpness and alignment with the dataset compared to the base OT-CFM model.

Limitations and Future Work

  • Invertibility Assumption: The method relies on the vector field being reversible. While this holds for many flow models, it limits generalization to non-reversible generative architectures.
  • Two-Stage Training: Currently, the flow model must be pre-trained before the corrector is trained. The authors suggest that training them simultaneously or as a single multi-output model could be a promising future direction.
  • Model Size: While the corrector is smaller than the base model, it still adds parameters and memory overhead.

In conclusion, Marginal Matching offers a mathematically sound and computationally cheap way to squeeze more performance out of flow matching models, making them even more competitive with state-of-the-art diffusion techniques.

Summary updated

Abstract

Flow matching models, ODE-based generative models, generate samples by gradually morphing a simple source distribution into a target distribution. However, these models still face challenges in optimally balancing computational efficiency with generation fidelity. While predictor-corrector schemes have been proven to help address this issue in diffusion models effectively, their application to flow matching models remains under-explored. In this work we present a novel training-based predictor-corrector framework for flow matching, which exploits the fact that the ODE can be easily reversed. Our key idea, termed Marginal Matching, is to correct samples at a particular time step in the forward ODE towards a matching marginal distribution of the backward ODE. This scheme proves to reduce a bound on the distance between the data distribution and the learned distribution, motivating two different implementations for the correction function. We show that our proposed method improves sample quality on CIFAR-10 and ImageNet-64, with minimal overhead in computation time.
BibTeX
@misc{bojan2026correcting,
  author = {Meital Bojan and Nadav Bojan Sellam and Ev Zisselman and Aviv Tamar},
  title = {{Correcting Flows with Marginal Matching}},
  year = {2026},
  eprint = {2611.23134},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.23134},
}
Figure