Abstract
Generative models like Diffusion and Flow Matching often fail to capture the full diversity of a target distribution within a small sampling budget. We present DiverseFlow, a training-free inference-time method that couples sample trajectories using Determinantal Point Processes (DPP). By maximizing the volume spanned by samples in a feature space, DiverseFlow effectively discovers multiple modes of ambiguous prompts and improves recall in complex image synthesis tasks without sacrificing quality.
Introduction
Generative models transform simple noise into complex data. While effective, the standard practice of independent and identically distributed (IID) sampling from the noise space does not guarantee diversity in the output space. Because the mapping from noise to image is non-linear, diverse starting points can still converge to the same narrow peak (mode) of the target distribution. This "mode collapse" in small batches forces users to generate dozens of samples to find desired variations, wasting time and compute.
Methodology
DiverseFlow modifies the sampling process without requiring retraining. It treats a batch of $K$ samples as a coupled system, where the velocity of each sample is influenced by a "diversity gradient."
- Determinantal Point Processes (DPP): Diversity is measured as the "volume" spanned by samples in a high-dimensional feature space (e.g., ViT features). Maximizing the determinant of the similarity kernel encourages orthogonality and thus diversity.
- Look-ahead Estimation: At each step, the algorithm predicts the final image $\hat{x}_1$ and adjusts the current trajectory to maximize the diversity of these predictions.
- Quality Constraints: To prevent artifacts, a constraint keeps the estimated source noise within high-probability regions of the original Gaussian distribution.
Results
DiverseFlow was evaluated on synthetic 2D densities, polysemous text prompts, and ImageNet-256.
Semantic Diversity
DiverseFlow excels at uncovering alternative meanings of ambiguous prompts. For the prompt "letter," IID sampling often produces only handwritten notes, whereas DiverseFlow discovers the typographic mode.
Similarly, for the prompt "boxer," DiverseFlow successfully generates both the dog breed and human athletes, whereas IID sampling frequently collapses to a single interpretation.
Distribution Coverage
On ImageNet-256, DiverseFlow improved Recall (a measure of distribution coverage) from 0.44 to 0.47 while maintaining high image quality. In synthetic tests with 10 modes, DiverseFlow consistently found all 10 modes with just 10 samples, compared to only ~6.5 for IID sampling.
Conclusion
DiverseFlow provides a robust, training-free solution to sample inefficiency. By mathematically grounding diversity in volume maximization, it allows for efficient discovery of diverse modes in both semantic and visual spaces, offering a compelling trade-off for resource-constrained generative tasks.
Summary updated