How to Fill the Optimum Set? Population Gradient Descent with Harmless Diversity

Chengyue Gong, Lemeng Wu, Qiang Liu

Abstract

Modern machine learning models, particularly overparameterized neural networks, often possess an infinite number of optimal solutions. While standard optimization typically converges to a single point, finding a diverse set of such optima is highly beneficial for tasks requiring variety or uncertainty estimation. We propose a population gradient descent framework that treats diversity as a secondary, "harmless" constraint. By updating particles only in directions that do not degrade the primary objective, our method efficiently explores the manifold of optima. We demonstrate superior performance in text-to-image generation, molecular modeling, and ensemble training.

The Challenge of Diversity in Optimization

In deep learning, the "optimum" is rarely a single point but rather a vast, low-dimensional manifold of solutions that all achieve minimal loss. Traditional "greedy" optimization methods find one such point and terminate. Generating variety usually requires multiple random restarts, which is computationally expensive and does not guarantee diversity. Alternatively, adding a diversity penalty often results in a trade-off where the quality of the primary objective is sacrificed for variety.

The central question addressed is: How can we populate the optimum set with diverse, representative points without compromising the main objective?

Methodology: Harmless Diversity

We frame this as a bi-level optimization problem where the primary goal is minimizing a loss function, and the secondary goal is maximizing a diversity score among a population of particles.

Harmless Update Rule

The core innovation is the Harmless Diversity Update. Instead of a simple weighted sum of gradients, we project the diversity gradient onto the subspace that does not interfere with the descent of the main loss. This ensures that diversity is "free"—it never pushes a particle "uphill" on the primary loss landscape.

Utility Functions: $F_{sum}$ vs. $F_{max}$

We explore two population management strategies:

  • $F_{sum}$: Minimizes the average loss of the population.
  • $F_{max}$: Focuses on the worst-performing particle. This approach is particularly effective as it allows high-performing particles to move freely along loss contours to maximize diversity, provided they remain better than the current worst particle.
Comparison of $F_{max}$ and $F_{sum}$ optimization dynamics across different loss landscapes.

Riesz s-energy

To measure diversity, we utilize Riesz s-energy (Newtonian energy) rather than standard variance. This physics-inspired metric imposes a stronger penalty on particles that are close together, preventing clumping and ensuring a more uniform spread across the solution space.

Key Results and Insights

The proposed framework was validated across diverse domains, including text-to-image (BigGAN, StyleGAN-v2), text-to-mesh, molecular conformation (GEOM-QM9), and ensemble learning (CIFAR-10).

Pareto Superiority

A critical finding is that our method achieves Pareto superiority over standard linear combination approaches. As shown in the figure below, the population gradient descent methods ($F_{sum}$ and $F_{max}$) consistently reach higher diversity levels at better (lower) loss scores compared to the traditional weighted-sum baseline.

Pareto front showing the trade-off between diversity and objective score for different optimization strategies.

Efficiency and Quality

  • Speed: Our gradient-based approach is significantly faster than evolutionary methods like MEGA, achieving comparable results in 10 minutes versus an hour.
  • Qualitative Variety: In generative tasks, the method produces visually distinct results (e.g., different hairstyles or accessories for a single prompt) where standard methods often collapse to near-identical outputs.
  • Robust Ensembles: For neural networks, the diverse populations lead to ensembles with improved Expected Calibration Error (ECE), providing more reliable uncertainty estimates.

Conclusion

By mathematically isolating "harmless" directions for diversity, we enable models to explore the manifold of optima without sacrificing performance. This framework provides a robust and efficient way to generate diverse solutions in high-dimensional spaces, with promising applications in generative AI, scientific modeling, and robotics.

Summary updated

Abstract

Although traditional optimization methods focus on finding a single optimal solution, most objective functions in modern machine learning problems, especially those in deep learning, often have multiple or infinite numbers of optima. Therefore, it is useful to consider the problem of finding a set of diverse points in the optimum set of an objective function. In this work, we frame this problem as a bi-level optimization problem of maximizing a diversity score inside the optimum set of the main loss function, and solve it with a simple population gradient descent framework that iteratively updates the points to maximize the diversity score in a fashion that does not hurt the optimization of the main loss. We demonstrate that our method can efficiently generate diverse solutions on a variety of applications, including text-to-image generation, text-to-mesh generation, molecular conformation generation and ensemble neural network training.
BibTeX
@inproceedings{gong2022how,
  author = {Chengyue Gong and Lemeng Wu and Qiang Liu},
  title = {{How to Fill the Optimum Set? Population Gradient Descent with Harmless Diversity}},
  booktitle = {International Conference on Machine Learning},
  year = {2022},
  url = {https://openreview.net/forum?id=6-kZc7ZQ4Ss},
}

Loading related content...

Figure