Abstract
Modern machine learning models, particularly overparameterized neural networks, often possess an infinite number of optimal solutions. While standard optimization typically converges to a single point, finding a diverse set of such optima is highly beneficial for tasks requiring variety or uncertainty estimation. We propose a population gradient descent framework that treats diversity as a secondary, "harmless" constraint. By updating particles only in directions that do not degrade the primary objective, our method efficiently explores the manifold of optima. We demonstrate superior performance in text-to-image generation, molecular modeling, and ensemble training.
The Challenge of Diversity in Optimization
In deep learning, the "optimum" is rarely a single point but rather a vast, low-dimensional manifold of solutions that all achieve minimal loss. Traditional "greedy" optimization methods find one such point and terminate. Generating variety usually requires multiple random restarts, which is computationally expensive and does not guarantee diversity. Alternatively, adding a diversity penalty often results in a trade-off where the quality of the primary objective is sacrificed for variety.
The central question addressed is: How can we populate the optimum set with diverse, representative points without compromising the main objective?
Methodology: Harmless Diversity
We frame this as a bi-level optimization problem where the primary goal is minimizing a loss function, and the secondary goal is maximizing a diversity score among a population of particles.
Harmless Update Rule
The core innovation is the Harmless Diversity Update. Instead of a simple weighted sum of gradients, we project the diversity gradient onto the subspace that does not interfere with the descent of the main loss. This ensures that diversity is "free"—it never pushes a particle "uphill" on the primary loss landscape.
Utility Functions: $F_{sum}$ vs. $F_{max}$
We explore two population management strategies:
- $F_{sum}$: Minimizes the average loss of the population.
- $F_{max}$: Focuses on the worst-performing particle. This approach is particularly effective as it allows high-performing particles to move freely along loss contours to maximize diversity, provided they remain better than the current worst particle.
Riesz s-energy
To measure diversity, we utilize Riesz s-energy (Newtonian energy) rather than standard variance. This physics-inspired metric imposes a stronger penalty on particles that are close together, preventing clumping and ensuring a more uniform spread across the solution space.
Key Results and Insights
The proposed framework was validated across diverse domains, including text-to-image (BigGAN, StyleGAN-v2), text-to-mesh, molecular conformation (GEOM-QM9), and ensemble learning (CIFAR-10).
Pareto Superiority
A critical finding is that our method achieves Pareto superiority over standard linear combination approaches. As shown in the figure below, the population gradient descent methods ($F_{sum}$ and $F_{max}$) consistently reach higher diversity levels at better (lower) loss scores compared to the traditional weighted-sum baseline.
Efficiency and Quality
- Speed: Our gradient-based approach is significantly faster than evolutionary methods like MEGA, achieving comparable results in 10 minutes versus an hour.
- Qualitative Variety: In generative tasks, the method produces visually distinct results (e.g., different hairstyles or accessories for a single prompt) where standard methods often collapse to near-identical outputs.
- Robust Ensembles: For neural networks, the diverse populations lead to ensembles with improved Expected Calibration Error (ECE), providing more reliable uncertainty estimates.
Conclusion
By mathematically isolating "harmless" directions for diversity, we enable models to explore the manifold of optima without sacrificing performance. This framework provides a robust and efficient way to generate diverse solutions in high-dimensional spaces, with promising applications in generative AI, scientific modeling, and robotics.
Summary updated