On the Closed-Form of Flow Matching: Generalization Does Not Arise from Target Stochasticity

Quentin Bertrand, Anne Gagneux, Mathurin Massias, Rémi Emonet

TL;DR

This paper challenges the popular theory that target stochasticity (noise in the training loss) is what allows flow matching models to generalize rather than memorize training data. By training models directly on a less stochastic, closed-form empirical target, the authors demonstrate that generalization actually arises because finite-capacity neural networks fail to approximate the complex, memorizing optimal velocity field early in the generation trajectory.

What Problem Does This Paper Address?

Deep generative models, such as diffusion and flow matching, excel at producing highly realistic samples. However, understanding why they generalize so effectively instead of simply memorizing the training set remains a foundational puzzle.

One prominent hypothesis, often termed "generalization through variance," suggests that the stochastic nature of the conditional flow matching loss acts as a regularizer that prevents memorization. Yet, analyzing the exact mathematical minimizer of this loss reveals a paradox: the true optimal empirical velocity field actually points directly toward individual training samples and diverges at the end of the trajectory. If a network were to learn this field perfectly, it would only replicate the training data.

The authors set out to resolve this paradox by answering a fundamental question: Does training on noisy, stochastic targets actually improve generalization in flow matching?

Methods

To isolate the effects of loss noise, the authors leverage the closed-form expression of the optimal empirical velocity field ($\hat{u}^\star$). They introduce Empirical Flow Matching (EFM), a novel training framework detailed in Algorithm 2.

Instead of regressing against the standard stochastic conditional target, EFM uses a mini-batch Monte Carlo approximation to regress directly against the deterministic, closed-form optimal velocity field. Crucially, the authors design an unbiased estimator that systematically includes the data point generating the current path variable, proving mathematically that it yields lower conditional variance than standard conditional flow matching (CFM).

Datasets

The authors validate their findings across several benchmarks:

  • Two-Moons: A standard 2D toy dataset used to contrast low-dimensional behavior against high-dimensional regimes.
  • CIFAR-10: A standard image dataset ($32 \times 32$ resolution) used for evaluating generative performance and memorization metrics.
  • CelebA: A dataset of celebrity faces cropped to $64 \times 64$ resolution.
  • Imagenette: A 10-class subset of ImageNet, spatially subsampled at multiple resolutions (from $8 \times 8$ to $256 \times 256$) to explicitly study how data dimensionality affects target stochasticity.
  • MNIST / Fashion-MNIST: Used in the appendix for additional verification of training dynamics.

Key Insights and Ideas

  • High Dimensions Kill Stochasticity: In low dimensions (like the 2D Two-Moons dataset), the optimal velocity field is a smooth average of many directions. However, as data dimensionality increases, the softmax weights in the closed-form velocity field collapse almost immediately. For real image datasets, the optimal field aligns with a single training sample's direction very early in the trajectory ($t < 0.2$), meaning the "stochastic" regime barely exists in practice.

    Dimension Dependence of Velocity Field Alignment
  • Generalization via Capacity Limits: Since the true optimal velocity field is highly complex and non-Lipschitz, a neural network with limited capacity naturally fails to approximate it perfectly. The authors identify two critical windows where the network fails to match the target: an early-time window ($t \approx 0.15$) and a late-time window ($t \to 1$).

  • Early Trajectory Determines Creativity: Using hybrid models that switch from the true optimal field to the learned network field at a threshold time $\tau$, the authors show that the first 20% of the generation trajectory ($t \le 0.2$) is where generalization happens. If the model follows the true memorizing path past $t = 0.4$, it completely loses its creativity and only outputs training samples.

Key Results

  • Deterministic Targets Improve Performance: Training with the less stochastic EFM target does not harm generalization. On the contrary, reducing target variance consistently yields modest but steady improvements in sample quality.
  • Stable and Faster Convergence: Across both CIFAR-10 and CelebA, increasing the number of Monte Carlo samples $M$ (making the target more deterministic) led to lower Fréchet Inception Distance (FID) scores earlier in the training process compared to vanilla CFM and Optimal Transport Flow Matching (OTCFM).

Limitations and Future Work

  • Empirical Focus: The study is largely empirical and evaluates the learned models primarily along their generation trajectories. It does not fully characterize how the velocity field behaves in unvisited regions of the state space.
  • Inductive Bias Interplay: The authors acknowledge that they did not deeply explore the interaction between the network's architectural inductive biases (e.g., the convolutional structures of U-Nets) and the optimization algorithms, which remains an active and challenging area of generative learning research.

Summary updated

Abstract

Modern deep generative models can now produce high-quality synthetic samples that are often indistinguishable from real training data. A growing body of research aims to understand why recent methods, such as diffusion and flow matching techniques, generalize so effectively. Among the proposed explanations are the inductive biases of deep learning architectures and the stochastic nature of the conditional flow matching loss. In this work, we rule out the noisy nature of the loss as a key factor driving generalization in flow matching. First, we empirically show that in high-dimensional settings, the stochastic and closed-form versions of the flow matching loss yield nearly equivalent losses. Then, using state-of-the-art flow matching models on standard image datasets, we demonstrate that both variants achieve comparable statistical performance, with the surprising observation that using the closed-form can even improve performance.
BibTeX
@inproceedings{bertrand2025closed,
  author = {Quentin Bertrand and Anne Gagneux and Mathurin Massias and Rémi Emonet},
  title = {{On the Closed-Form of Flow Matching: Generalization Does Not Arise from Target Stochasticity}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2025},
  url = {https://openreview.net/forum?id=kVz9uvqUna},
}
Figure