On the Closed-Form of Flow Matching: Generalization Does Not Arise from Target Stochasticity

Quentin Bertrand, Anne Gagneux, Mathurin Massias, Rémi Emonet

Abstract: Modern deep generative models can now produce high-quality synthetic samples that are often indistinguishable from real training data. A growing body of research aims to understand why recent methods -- such as diffusion and flow matching techniques -- generalize so effectively. Among the proposed explanations are the inductive biases of deep learning architectures and the stochastic nature of the conditional flow matching loss. In this work, we rule out the latter -- the noisy nature of the loss -- as a primary contributor to generalization in flow matching. First, we empirically show that in high-dimensional settings, the stochastic and closed-form versions of the flow matching loss yield nearly equivalent losses. Then, using state-of-the-art flow matching models on standard image datasets, we demonstrate that both variants achieve comparable statistical performance, with the surprising observation that using the closed-form can even improve performance.

Introduction: The Memorization Paradox

Deep generative models, such as Diffusion and Flow Matching, present a theoretical puzzle known as the "memorization paradox." If these models were to perfectly minimize their training loss, they should theoretically learn to reconstruct training samples exactly, failing to generate novel data. However, in practice, they generalize remarkably well.

A leading hypothesis suggests that target stochasticity is the key mechanism for this generalization. In standard flow matching, the model regresses against a velocity vector pointing toward a single random training sample. The theory, notably explored by Vastola (2025), was that this "noise" in the loss function acts as a regularizer, preventing the model from over-committing to any one sample and thus forcing it to learn the broader distribution. This paper asks: Is that actually true?

Methods: Empirical Flow Matching (EFM)

To investigate this, the authors analyze the optimal velocity field ($u^*$), the theoretical field that maps a starting noise distribution exactly to the empirical training data distribution. They leverage a closed-form formula for this optimal field, showing it is a weighted average of directions toward all training points, where the weights are determined by a softmax of distances.

Based on this analysis, they introduce Empirical Flow Matching (EFM). Instead of the "vanilla" approach of regressing against one random sample, EFM regresses against a Monte Carlo estimate of the full, closed-form optimal velocity field using $M$ samples. As $M$ increases, the training target becomes more deterministic and less "noisy." If stochasticity were the driver of generalization, EFM should perform worse as $M$ increases; however, the authors find the opposite.

The High-Dimensional Collapse of Stochasticity

The most striking insight is that in high dimensions (like 32x32 images), the "stochastic" target isn't actually very stochastic. Because of the "curse of dimensionality," the optimal velocity field points almost exclusively toward the single nearest training sample very early in the process ($t > 0.2$). The "noise" that was supposed to drive generalization effectively vanishes in high-dimensional space.

Analysis of target stochasticity across dimensions and time.

As shown in the figure above, for high-dimensional data like CIFAR-10, the cosine similarity between the training target and the optimal field quickly approaches 1. This means the training signal is nearly deterministic for the majority of the trajectory, yet the models still generalize perfectly well. This suggests that the stochasticity of the loss is a red herring.

Generalization via Network "Failure"

If stochasticity isn't the cause, what is? The authors argue that generalization happens because the neural network fails to approximate the optimal velocity field. Specifically, the network lacks the capacity to learn the highly complex, non-smooth "jumps" required to perfectly memorize every training point. The network's inductive bias toward smooth functions acts as the true regularizer.

Furthermore, by using hybrid trajectories, the authors found that generalization is "decided" very early ($t \approx 0.15$). If a model follows the perfect memorization path past this point, it loses the ability to be "creative," regardless of what it does later. This highlights the importance of the early stages of the generative process in establishing the model's ability to produce novel samples.

Key Results and Performance

The empirical results strongly support the claim that deterministic training is at least as effective as stochastic training:

  • Performance Gains: On CIFAR-10 and CelebA, Empirical Flow Matching (EFM) consistently matched or outperformed Vanilla Flow Matching (CFM). For example, using $M=1000$ samples for the target estimate led to lower (better) FID scores across both Inception and DINOv2 metrics.
  • Stability and Convergence: Increasing the number of samples $M$ in the EFM target led to more stable training and faster convergence in terms of epochs, suggesting that a cleaner training signal is beneficial.
  • Dataset Size and Memorization: The study also highlights that while low Test FID scores are often used as a proxy for generalization, they can be misleading on small datasets where memorization is easier. As dataset size increases, the gap between the learned field and the memorizing field grows, indicating a transition to true generalization.

Conclusion and Future Work

This work rules out target stochasticity as a primary contributor to generalization in flow matching. Instead, the results point toward the inductive biases of deep learning architectures as the source of generative success. While the study provides a clear empirical debunking, it leaves open the challenge of fully characterizing these inductive biases and understanding how different architectures (e.g., Transformers vs. U-Nets) interact with the velocity field approximation. Understanding off-trajectory behavior remains a key frontier for explaining the full "magic" of modern generative models.

Summary updated

Abstract

Modern deep generative models can now produce high-quality synthetic samples that are often indistinguishable from real training data. A growing body of research aims to understand why recent methods -- such as diffusion and flow matching techniques -- generalize so effectively. Among the proposed explanations are the inductive biases of deep learning architectures and the stochastic nature of the conditional flow matching loss. In this work, we rule out the latter -- the noisy nature of the loss -- as a primary contributor to generalization in flow matching. First, we empirically show that in high-dimensional settings, the stochastic and closed-form versions of the flow matching loss yield nearly equivalent losses. Then, using state-of-the-art flow matching models on standard image datasets, we demonstrate that both variants achieve comparable statistical performance, with the surprising observation that using the closed-form can even improve performance.
BibTeX
@inproceedings{bertrand2025closed,
  author = {Quentin Bertrand and Anne Gagneux and Mathurin Massias and Rémi Emonet},
  title = {{On the Closed-Form of Flow Matching: Generalization Does Not Arise from Target Stochasticity}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {2025},
  url = {https://openreview.net/forum?id=kVz9uvqUna},
}
Figure