Tracing Generated Samples to Training-Data Clusters in Flow-Matching Models

Rania Briq, Ohad Fried, Michael Kamp, Stefan KesselheimarXiv 2026 · arXiv

TL;DR

This paper introduces a method for tracing generated images back to specific training-data clusters in flow-matching models. By leveraging the analytical closed-form solution of rectified flows, the authors track how training clusters influence a sample along its generation trajectory and validate their approach using computationally heavy "leave-one-cluster-out" retrained models.

What Problem Does This Paper Address?

Data attribution—identifying which training examples most heavily influenced a particular generated output—is crucial for interpretability, copyright resolution, and mitigating unwanted model behaviors. While well-studied in classification, attribution is notoriously difficult in generative models.

In Flow Matching (FM) models, images are generated by integrating a learned velocity field along a trajectory. If you remove a training sample, the entire velocity field shifts, and this change propagates dynamically all the way to the final image. Because local velocity perturbations don't map linearly to final image changes, traditional gradient-based attribution methods often fall short. Furthermore, latent flow-matching models are highly stable; removing a single training image rarely changes the output identity. To combat this, the authors shift the focus from individual samples to cluster-level attribution, asking: Which semantic group of training images is responsible for this generated output?

Methods

The authors propose a hybrid analytical–learned approach to estimate cluster influence without needing to retrain the model for every query.

  1. Leave-One-Cluster-Out (LOCO) Oracles: To establish a ground truth for evaluation, the authors partition the training data into $K=12$ clusters. They then train $K$ separate "counterfactual" models, each completely omitting one cluster. The true influence of a cluster is measured by the Root Mean Square (RMS) distance between the full model's output and the LOCO model's output.
  2. Closed-Form (CF) Trajectory Attribution: Rectified flows possess an elegant mathematical property: their optimal velocity field can be expressed as a linear combination of training samples weighted by a softmax function. The authors use this to define two training-free metrics evaluated along the generation path:
  3. Closed-form cluster mass: The cumulative softmax weight a cluster contributes to the velocity field across the entire trajectory.
  4. Closed-form deletion: A metric quantifying how much the local velocity field is perturbed if a specific cluster's weights are zeroed out and renormalized.
Figure 1: Top: Overview of the proposed attribution approach based on rectified-flow closed-form solution (CF). We train counterfactual leave-one-cluster-out (LOCO) models for evaluation. CF- based attribution assigns an attribution score to each cluster by evaluating its contribution along the trajectory. Bottom: Full-model output and K = 12 counterfactual images generated by the LOCO models. −Ck denotes the model retrained without cluster Ck. C9 yields the highest RMS, making it the winning (most influential) cluster.

Datasets

The experiments are conducted on the CelebA-HQ dataset using its standard 28k training split. To define the clusters, the authors extract features using DINOv2 (and CLIP for an ablation study) and apply $k$-means clustering to group the celebrity faces into 12 distinct semantic clusters.

Key Insights and Ideas

  • The Latent Space Dictates Attribution Quality: The geometry of the model's latent space heavily influences how well we can track data lineage. Models operating in a highly structured semantic space (like a DINOv2-based Representation Autoencoder, or RAE) yield significantly better attribution accuracy than those operating in standard VAE latent spaces, which prioritize pixel reconstruction over semantic structure.
  • Training Stochasticity is a Major Confounder: One of the most surprising findings was that models trained with different random seeds on the exact same data show massive variations in their generation endpoints. In fact, different seeds agreed on the "most influential cluster" only 25% of the time. This highlights that counterfactual attribution is deeply affected by training randomness, necessitating multi-seed averaging to find a stable pseudo-ground-truth.
  • Semantic Similarity is a Formidable Baseline: Simply measuring the distance from the generated image's endpoint to the cluster centroids in DINOv2 space performs remarkably well, often beating complex trajectory-tracking methods in predicting the absolute top influential cluster.

Key Results

The authors evaluated their methods against single-seed and multi-seed oracles using Top-1 agreement (how often the method guesses the most influential cluster) and Spearman rank correlation.

  • For DINOv2-based RAE models: The simple DINOv2 centroid baseline achieved the highest Top-1 agreement at 51.6% against the multi-seed oracle. However, the Closed-form mass and Closed-form deletion methods achieved the highest Spearman rank correlation (0.373), proving they are superior at ranking the overall order of cluster importance across the entire dataset without requiring any model gradients or retraining.
Table 1: Attribution performance for DINOv2-based RAE models evaluated for N = 512 generated samples against single-seed and averaged multi-seed oracles. We omit nearest-member in multi-seed due to lack of clear advantage and the associated computational cost.
  • Latent Space Comparison: When switching to a VAE-based framework (SiT), overall attribution performance dropped sharply across all methods, confirming that standard VAE spaces are less optimal for distance-based data tracing.
Attribution performance for the VAE-based flow-matching model.

Limitations and Future Work

While this work lays a solid foundation for flow-matching attribution, it has a few key limitations:

  • Linear Paths Only: The elegant closed-form solution used here is uniquely available for rectified flows with linear paths; it does not easily extend to more complex, non-linear diffusion paths.
  • Coarse Granularity: The method operates at the cluster level ($K=12$). Finer-grained, sample-level attribution remains an open challenge due to the extreme stability of flow-matching models under single-sample deletion.
  • Domain Constraints: The evaluation was restricted to the image domain (specifically human faces). Testing how these dynamics transfer to text, audio, or broader image datasets is left for future work.

Summary updated

Abstract

Understanding which training samples influence a generated image is an important problem in generative modeling. In flow matching, training samples influence the generated image through the velocity field along the generation trajectory. Removing samples to examine their counterfactual influence changes the velocity field, and the resulting effect on the final image depends on how the change propagates through the trajectory. Consequently, local changes in the velocity field do not necessarily predict the final counterfactual effect. This work investigates attribution in flow-matching models through a hybrid analytical--learned approach, and uses it to derive trajectory-based attribution scores at the cluster level. We evaluate these attribution scores using independently retrained leave-one-cluster-out (LOCO) models, and compare with several attribution baselines using two different flow-matching latent spaces. Our experiments show that semantic similarity constitutes a strong baseline, while the closed-form trajectory-based attribution is competitive in some metrics without requiring counterfactual retraining or model gradients. Our results show that attribution in flow matching depends not only on semantic similarity to training samples, but also on the latent representation, trajectory dynamics, and how influence is propagated to the final output.
BibTeX
@misc{briq2026tracing,
  author = {Rania Briq and Ohad Fried and Michael Kamp and Stefan Kesselheim},
  title = {{Tracing Generated Samples to Training-Data Clusters in Flow-Matching Models}},
  year = {2026},
  eprint = {2608.30081},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2608.30081},
}
Figure