In the high-stakes world of generative AI, we usually treat model selection as a "winner-takes-all" contest. We train a dozen architectures, calculate their Fréchet Inception Distance (FID) or Kernel Inception Distance (KID), and crown the single best performer as our production model. But what if the "best" model isn't a single model at all, but a cocktail of several?
A new paper presented at ICLR 2025, "Be More Diverse than the Most Diverse," argues exactly that. By framing model selection as a multi-armed bandit problem, the authors demonstrate that an optimal mixture of generative models can significantly outperform even the best individual model in both diversity and quality.
TL;DR
Instead of picking the single best generative model from a pool, this paper proves that a weighted mixture of models often achieves superior diversity and quality scores. The authors introduce Mixture-UCB, a multi-armed bandit algorithm that efficiently finds these optimal mixture weights while minimizing the number of samples wasted on sub-optimal models.
What Problem Does This Paper Address?
Current evaluation metrics for generative models, like FID and KID, reward two things: fidelity (how realistic the samples look) and diversity (how well the samples cover the entire data distribution).
The standard approach is to pick the one model that maximizes these scores. However, the authors point out a mathematical oversight: these evaluation scores are often convex functions of the model's distribution. In optimization terms, this means the "best" score is frequently found in the middle of the search space (a mixture) rather than at the corners (individual models).
For example, one model might be great at generating "realistic faces" but poor at "diverse accessories," while another model has the opposite strengths. A mixture can cover the blind spots of both. The challenge is the "chicken and egg" problem: to find the optimal mixture weights, you need to sample the models; but to sample efficiently without wasting time and money, you already need to know which models are worth sampling.
Methods
The authors frame the search for the perfect mixture as a Multi-Armed Bandit (MAB) problem. In this setup, each generative model is an "arm." The goal is to learn the optimal probability distribution (mixture weights) over these arms to minimize a loss function (the evaluation score).
Key Innovations:
- Quadratic Score Formulation: They show that popular metrics like KID and Rényi Kernel Entropy (RKE) can be written as quadratic functions of the mixture weights. This allows them to treat the task as a quadratic programming problem.
- Mixture-UCB-CAB: This algorithm treats the set of all possible mixture weights as an infinite set of arms (a "continuum-armed bandit"). It uses an Upper Confidence Bound (UCB) strategy to explore the weight space, prioritizing mixtures that might be optimal based on current data.
- Mixture-UCB-OGD: A more computationally efficient version that uses Online Gradient Descent to update weights. While Mixture-UCB-CAB requires solving a quadratic program at every step, OGD simply follows the gradient of the loss, making it much faster for large-scale applications.
Unlike traditional UCB, which eventually converges to pulling only the single best arm, Mixture-UCB is designed to converge to a stable, non-degenerate mixture of multiple arms.
Datasets
The researchers validated their approach across several domains:
- Image Generation: FFHQ (faces), LSUN-Bedroom, and AFHQ Cat. They used models like StyleGAN-XL, LDM (Latent Diffusion), and Efficient-VDVAE.
- Text-to-Image: Using state-of-the-art models like Stable Diffusion XL, Kandinsky 3.0, and PixArt-α to generate specific prompts (e.g., "Red bird, cartoon style").
- Text Generation: The OpenLLMText dataset, featuring outputs from GPT2-XL, LLaMA-7B, and PaLM.
Key Insights and Ideas
- The Diversity Boost: The most striking insight is that mixing models creates "emergent diversity." As seen in Figure 2, a mixture of three text-to-image models produces a wider variety of "red birds" than any single model could manage alone.
- Convexity is Key: Because metrics like KID are convex, the optimal solution is mathematically likely to be a mixture. The paper provides the theoretical framework to exploit this property.
- Efficiency Matters: In real-world settings, generating samples (especially from large LLMs or diffusion models) is expensive. The Mixture-UCB approach identifies the optimal mixture using significantly fewer samples than naive grid search or random sampling.
Key Results
The quantitative improvements are significant. In the FFHQ face generation experiment (Figure 1), the best individual model (StyleNAT) achieved an FID of 185.07. The optimal mixture found by the algorithm achieved a much better FID of 170.11.
Further results include:
- Convergence: As shown in Figure 3, the Mixture-UCB algorithms (CAB and OGD) consistently converge to lower KID scores faster than baselines like "Vanilla-UCB" (which only looks for one best arm) or "Successive Halving."
- Text Diversity: For text generation, the mixture improved the RKE diversity score from 69.3 (best single model) to 75.2.
- Quality-Diversity Trade-off: By combining RKE (diversity) with Precision/Density (quality) in the loss function, the algorithm can find mixtures that are both highly diverse and high-quality, effectively filtering out "trash" models that offer diversity at the cost of realism.
Limitations and Future Work
While the paper provides a strong theoretical and empirical foundation, a few areas remain open:
- Regret Bounds for OGD: While the authors proved a regret bound for the CAB version of the algorithm, a formal bound for the faster OGD version is still needed.
- Quality Metrics: The authors note that "linear" quality scores (like Density) are still optimized by single models. The mixture approach is most powerful when diversity is a primary goal.
- Computational Overhead: Although OGD is fast, calculating kernel-based metrics (like KID) on the fly during the bandit process still requires significant memory as the number of samples grows.
The project code is available at https://github.com/Rezaei-Parham/Mixture-UCB.
Summary updated