TL;DR
This paper introduces a method for tracing generated images back to specific training-data clusters in flow-matching models. By leveraging the analytical closed-form solution of rectified flows, the authors track how training clusters influence a sample along its generation trajectory and validate their approach using computationally heavy "leave-one-cluster-out" retrained models.
What Problem Does This Paper Address?
Data attribution—identifying which training examples most heavily influenced a particular generated output—is crucial for interpretability, copyright resolution, and mitigating unwanted model behaviors. While well-studied in classification, attribution is notoriously difficult in generative models.
In Flow Matching (FM) models, images are generated by integrating a learned velocity field along a trajectory. If you remove a training sample, the entire velocity field shifts, and this change propagates dynamically all the way to the final image. Because local velocity perturbations don't map linearly to final image changes, traditional gradient-based attribution methods often fall short. Furthermore, latent flow-matching models are highly stable; removing a single training image rarely changes the output identity. To combat this, the authors shift the focus from individual samples to cluster-level attribution, asking: Which semantic group of training images is responsible for this generated output?
Methods
The authors propose a hybrid analytical–learned approach to estimate cluster influence without needing to retrain the model for every query.
- Leave-One-Cluster-Out (LOCO) Oracles: To establish a ground truth for evaluation, the authors partition the training data into $K=12$ clusters. They then train $K$ separate "counterfactual" models, each completely omitting one cluster. The true influence of a cluster is measured by the Root Mean Square (RMS) distance between the full model's output and the LOCO model's output.
- Closed-Form (CF) Trajectory Attribution: Rectified flows possess an elegant mathematical property: their optimal velocity field can be expressed as a linear combination of training samples weighted by a softmax function. The authors use this to define two training-free metrics evaluated along the generation path:
- Closed-form cluster mass: The cumulative softmax weight a cluster contributes to the velocity field across the entire trajectory.
- Closed-form deletion: A metric quantifying how much the local velocity field is perturbed if a specific cluster's weights are zeroed out and renormalized.
Datasets
The experiments are conducted on the CelebA-HQ dataset using its standard 28k training split. To define the clusters, the authors extract features using DINOv2 (and CLIP for an ablation study) and apply $k$-means clustering to group the celebrity faces into 12 distinct semantic clusters.
Key Insights and Ideas
- The Latent Space Dictates Attribution Quality: The geometry of the model's latent space heavily influences how well we can track data lineage. Models operating in a highly structured semantic space (like a DINOv2-based Representation Autoencoder, or RAE) yield significantly better attribution accuracy than those operating in standard VAE latent spaces, which prioritize pixel reconstruction over semantic structure.
- Training Stochasticity is a Major Confounder: One of the most surprising findings was that models trained with different random seeds on the exact same data show massive variations in their generation endpoints. In fact, different seeds agreed on the "most influential cluster" only 25% of the time. This highlights that counterfactual attribution is deeply affected by training randomness, necessitating multi-seed averaging to find a stable pseudo-ground-truth.
- Semantic Similarity is a Formidable Baseline: Simply measuring the distance from the generated image's endpoint to the cluster centroids in DINOv2 space performs remarkably well, often beating complex trajectory-tracking methods in predicting the absolute top influential cluster.
Key Results
The authors evaluated their methods against single-seed and multi-seed oracles using Top-1 agreement (how often the method guesses the most influential cluster) and Spearman rank correlation.
- For DINOv2-based RAE models: The simple DINOv2 centroid baseline achieved the highest Top-1 agreement at 51.6% against the multi-seed oracle. However, the Closed-form mass and Closed-form deletion methods achieved the highest Spearman rank correlation (0.373), proving they are superior at ranking the overall order of cluster importance across the entire dataset without requiring any model gradients or retraining.
- Latent Space Comparison: When switching to a VAE-based framework (SiT), overall attribution performance dropped sharply across all methods, confirming that standard VAE spaces are less optimal for distance-based data tracing.
Limitations and Future Work
While this work lays a solid foundation for flow-matching attribution, it has a few key limitations:
- Linear Paths Only: The elegant closed-form solution used here is uniquely available for rectified flows with linear paths; it does not easily extend to more complex, non-linear diffusion paths.
- Coarse Granularity: The method operates at the cluster level ($K=12$). Finer-grained, sample-level attribution remains an open challenge due to the extreme stability of flow-matching models under single-sample deletion.
- Domain Constraints: The evaluation was restricted to the image domain (specifically human faces). Testing how these dynamics transfer to text, audio, or broader image datasets is left for future work.
Summary updated