Scaling Group Inference for Diverse and High-Quality Generation

Gaurav Parmar, Or Patashnik, Daniil Ostashev, Kuan-Chieh Wang, Kfir Aberman, Srinivasa Narasimhan, Jun-Yan Zhu

Abstract

Generative models typically sample outputs independently, and recent inference-time guidance and scaling algorithms focus on improving the quality of individual samples. However, in real-world applications, users are often presented with a set of multiple images (e.g., 4-8) for each prompt, where independent sampling tends to lead to redundant results, limiting user choices and hindering idea exploration. In this work, we introduce a scalable group inference method that improves both the diversity and quality of a group of samples. We formulate group inference as a quadratic integer assignment problem: candidate outputs are modeled as graph nodes, and a subset is selected to optimize sample quality (unary term) while maximizing group diversity (binary term). To substantially improve runtime efficiency, we progressively prune the candidate set using intermediate predictions, allowing our method to scale up to large candidate sets. Extensive experiments show that our method significantly improves group diversity and quality compared to independent sampling baselines and recent inference algorithms. Our framework generalizes across a wide range of tasks, including text-to-image, image-to-image, image prompting, and video generation, enabling generative models to treat multiple outputs as cohesive groups rather than independent samples.

Introduction

Current state-of-the-art diffusion models like Midjourney or DALL-E typically provide users with a grid of images for a single prompt. Ideally, this set should offer a diverse range of interpretations, styles, and compositions to facilitate creative exploration. However, standard Independent and Identically Distributed (I.I.D.) sampling often results in redundancy. Because the model converges on the most probable interpretation of a prompt, the resulting images can be nearly identical, failing to provide the variety users expect.

While significant research has focused on improving the quality of individual generations through scaling laws and guidance techniques, the "group dynamics" of generation have been largely overlooked. The challenge lies in generating a set of images that is both high-quality and diverse without incurring prohibitive computational costs.

Methodology

The proposed Scalable Group Inference framework treats image set selection as a global optimization problem rather than a series of independent events.

Quadratic Integer Programming (QIP) Formulation

The selection process is modeled as a QIP problem. Given a large pool of $M$ initial candidates, the goal is to select a subset of size $K$ that maximizes a combined objective:

  • Unary Term (Quality): Quantifies individual sample alignment with the prompt, typically using metrics like CLIPScore.
  • Binary Term (Diversity): Quantifies the distance between pairs of images using high-level features (e.g., DINOv2).

Progressive Pruning

Generating a large pool of $M$ candidates (e.g., $M=128$) through the full diffusion process is computationally expensive. To address this, the authors introduce progressive pruning. This technique leverages the observation that intermediate $x_0$ estimates during the denoising process are reliable predictors of the final image's characteristics. By periodically generating "previews," calculating quality/diversity scores, and pruning the lowest-performing candidates, the algorithm focuses computational resources on the most promising samples. This reduces the complexity from $O(MT)$ to approximately $O(M + KT)$, where $T$ is the number of timesteps.

Experimental Evaluation

The method was evaluated across diverse benchmarks:

  • GenEval: For complex text-to-image alignment.
  • COCO 2017: For depth-conditioned image-to-image tasks.
  • DreamBooth: For personalized image generation.
  • User Studies: Over 23,000 preference judgments were collected via Amazon Mechanical Turk to validate diversity and quality improvements.

Results and Discussion

The Scalable Group Inference method demonstrates a significant improvement in the quality-diversity tradeoff.

Qualitative comparison across various prompts showing how Group Inference produces more diverse and high-quality sets compared to standard I.I.D. sampling.

Efficiency and Performance

Progressive pruning achieved speedups of 49% to 73% compared to full denoising of the candidate pool, with no significant loss in output quality. Quantitative results show that as the initial candidate pool size $M$ increases, the method consistently finds better Pareto-optimal sets than baselines like Particle Guidance or varying Classifier-Free Guidance (CFG).

Qualitative Variety

As shown in the visual comparisons, the method successfully explores different poses, colors, and camera angles. For instance, in the "spaghetti couple dancing" prompt, I.I.D. sampling often produces very similar poses, whereas Group Inference selects distinct artistic interpretations. Similar gains are seen in technical domains, such as watch sketches, and natural scenes like potted plants.

Model Agnosticism

A key strength of this approach is its flexibility. It functions as a selection layer that can be applied to various architectures (e.g., FLUX.1, Stable Diffusion 3) and modalities, including video generation. Furthermore, it supports non-differentiable scoring functions, such as those provided by Large Language Models (LLMs).

Conclusion

Scalable Group Inference effectively addresses the redundancy issue in multi-image generation. By treating a set of outputs as a cohesive group and utilizing progressive pruning for efficiency, the method provides users with higher quality and greater variety. Future work may involve developing "proposer" models that can more effectively seed the initial candidate pool with diverse latent representations, further enhancing the system's exploratory capabilities.

Summary updated

Abstract

Generative models typically sample outputs independently, and recent inference-time guidance and scaling algorithms focus on improving the quality of individual samples. However, in real-world applications, users are often presented with a set of multiple images (e.g., 4-8) for each prompt, where independent sampling tends to lead to redundant results, limiting user choices and hindering idea exploration. In this work, we introduce a scalable group inference method that improves both the diversity and quality of a group of samples. We formulate group inference as a quadratic integer assignment problem: candidate outputs are modeled as graph nodes, and a subset is selected to optimize sample quality (unary term) while maximizing group diversity (binary term). To substantially improve runtime efficiency, we progressively prune the candidate set using intermediate predictions, allowing our method to scale up to large candidate sets. Extensive experiments show that our method significantly improves group diversity and quality compared to independent sampling baselines and recent inference algorithms. Our framework generalizes across a wide range of tasks, including text-to-image, image-to-image, image prompting, and video generation, enabling generative models to treat multiple outputs as cohesive groups rather than independent samples.
BibTeX
@inproceedings{parmar2026scaling,
  author = {Gaurav Parmar and Or Patashnik and Daniil Ostashev and Kuan-Chieh Wang and Kfir Aberman and Srinivasa Narasimhan and Jun-Yan Zhu},
  title = {{Scaling Group Inference for Diverse and High-Quality Generation}},
  booktitle = {International Conference on Learning Representations},
  year = {2026},
  url = {https://openreview.net/forum?id=IyTNxjTuWT},
}
Figure