DiverseFlow: Sample-Efficient Diverse Mode Coverage in Flows

Mashrur M. MorshedSubmission to ICLR 2025 · Withdrawn

Abstract

Generative models like Diffusion and Flow Matching often fail to capture the full diversity of a target distribution within a small sampling budget. We present DiverseFlow, a training-free inference-time method that couples sample trajectories using Determinantal Point Processes (DPP). By maximizing the volume spanned by samples in a feature space, DiverseFlow effectively discovers multiple modes of ambiguous prompts and improves recall in complex image synthesis tasks without sacrificing quality.

Introduction

Generative models transform simple noise into complex data. While effective, the standard practice of independent and identically distributed (IID) sampling from the noise space does not guarantee diversity in the output space. Because the mapping from noise to image is non-linear, diverse starting points can still converge to the same narrow peak (mode) of the target distribution. This "mode collapse" in small batches forces users to generate dozens of samples to find desired variations, wasting time and compute.

Methodology

DiverseFlow modifies the sampling process without requiring retraining. It treats a batch of $K$ samples as a coupled system, where the velocity of each sample is influenced by a "diversity gradient."

  1. Determinantal Point Processes (DPP): Diversity is measured as the "volume" spanned by samples in a high-dimensional feature space (e.g., ViT features). Maximizing the determinant of the similarity kernel encourages orthogonality and thus diversity.
  2. Look-ahead Estimation: At each step, the algorithm predicts the final image $\hat{x}_1$ and adjusts the current trajectory to maximize the diversity of these predictions.
  3. Quality Constraints: To prevent artifacts, a constraint keeps the estimated source noise within high-probability regions of the original Gaussian distribution.

Results

DiverseFlow was evaluated on synthetic 2D densities, polysemous text prompts, and ImageNet-256.

Semantic Diversity

DiverseFlow excels at uncovering alternative meanings of ambiguous prompts. For the prompt "letter," IID sampling often produces only handwritten notes, whereas DiverseFlow discovers the typographic mode.

Comparison of IID (left) vs. DiverseFlow (right) for the prompt "letter". DiverseFlow discovers the typographic mode.

Similarly, for the prompt "boxer," DiverseFlow successfully generates both the dog breed and human athletes, whereas IID sampling frequently collapses to a single interpretation.

DiverseFlow results for the prompt "boxer", capturing both the athlete and the dog breed modes.

Distribution Coverage

On ImageNet-256, DiverseFlow improved Recall (a measure of distribution coverage) from 0.44 to 0.47 while maintaining high image quality. In synthetic tests with 10 modes, DiverseFlow consistently found all 10 modes with just 10 samples, compared to only ~6.5 for IID sampling.

Conclusion

DiverseFlow provides a robust, training-free solution to sample inefficiency. By mathematically grounding diversity in volume maximization, it allows for efficient discovery of diverse modes in both semantic and visual spaces, offering a compelling trade-off for resource-constrained generative tasks.

Summary updated

Abstract

Many real-world applications of flow generative models desire a diverse set of samples covering multiple modes of the target distribution. However, the predominant approach for obtaining diverse sets is not sample-efficient, as it involves independently obtaining many samples from the source distribution and mapping them through the flow until the desired mode coverage is achieved. As an alternative to repeated sampling, we introduce DiverseFlow---a training-free, inference-time approach to improve the diversity of flow models. Our key idea is to employ a determinantal point process to induce a coupling between the samples and drive sample diversity under a fixed sampling budget. We demonstrate the efficacy of DiverseFlow for tasks where sample efficient diversity is highly desirable---text-guided image generation with polysemous words, inverse problems like large-hole inpainting, and class-conditional image synthesis.
BibTeX
@misc{morshed2025diverseflow,
  author = {Mashrur M. Morshed},
  title = {{DiverseFlow: Sample-Efficient Diverse Mode Coverage in Flows}},
  year = {2025},
  eprint = {2504.07894},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2504.07894},
}
Figure