Abstract
In many real-world optimization problems, the location of multiple optima is often required in a search space. Evaluating solutions frequently involves thousands of fitness function evaluations, a process that is both time-consuming and expensive. Standard Particle Swarm Optimization (PSO) faces significant challenges when a large number of evaluations are required. This study proposes a model-assisted PSO (PSO-GP) that utilizes a Gaussian Process (GP) approximation model as a surrogate for the real fitness function. Training datasets for the GP are generated from the first two generations of particle information and are dynamically updated by replacing the worst-performing data points. Numerical simulations on 20-dimensional complex multimodal functions demonstrate that PSO-GP is significantly more efficient than standard PSO, achieving comparable accuracy with a fraction of the computational cost.
Introduction
In advanced engineering fields such as aerospace design or structural physics, optimization often relies on high-fidelity simulations like Computational Fluid Dynamics (CFD) or Finite Element Analysis (FEA). While these simulations provide accurate "fitness" evaluations for a given design, a single run can take hours or even days on high-performance computing clusters.
Particle Swarm Optimization (PSO) is a popular metaheuristic for navigating complex, multimodal landscapes—areas with numerous local peaks and valleys. However, the iterative nature of PSO typically demands thousands of function evaluations to converge on a global optimum. In the context of expensive simulations, this requirement makes standard PSO computationally prohibitive.
The PSO-GP Framework
To address this bottleneck, we introduce PSO-GP, a hybrid framework that integrates Gaussian Process (GP) surrogate models into the PSO search process. The primary objective is to substitute expensive real-world evaluations with a "cheap" mathematical approximation.
Gaussian Process Surrogates
Unlike traditional Artificial Neural Networks (ANNs), which often require extensive hyperparameter tuning and large static datasets, Gaussian Processes offer a robust Bayesian framework. GPs can learn the underlying shape of a fitness landscape from a relatively small set of data points while providing a measure of uncertainty for their predictions. This allows the algorithm to "understand" where its map of the search space is reliable and where it needs more information.
Two-Cycle Optimization Strategy
The PSO-GP algorithm operates through a dual-cycle structure:
- Inner Cycle: The swarm performs its standard "flight" through the search space. However, instead of querying the expensive real-world function, it uses the GP model to predict fitness values. This step is computationally instantaneous.
- Outer Cycle: Once the swarm identifies the most promising candidate based on the GP's prediction, that specific point is evaluated using the actual, high-fidelity function. This ensures the search remains grounded in reality.
Dynamic Dataset Renewal
A static surrogate model quickly becomes inaccurate as the search moves into new regions of the space. PSO-GP employs a dynamic update strategy to maintain model fidelity. The training dataset is initialized using particles from the first two generations. As the search progresses, the algorithm sorts the training points and replaces the worst-performing data with new, high-quality evaluations found during the outer cycle. This ensures the GP remains most accurate in the regions where the global optimum is likely to reside.
Experimental Results
The performance of PSO-GP was benchmarked against standard PSO using three 20-dimensional multimodal functions: Ackley, Griewank, and Rastrigin. These functions are designed to simulate the "traps" found in real-world engineering problems.
The results revealed a dramatic increase in efficiency:
- Ackley (20D): PSO-GP achieved convergence with approximately 14,817 evaluations, an 8x improvement over standard PSO.
- Griewank (20D): The algorithm required only 4,711 evaluations, making it 16x more efficient.
- Rastrigin (20D): In the most significant result, PSO-GP converged in just 867 evaluations, compared to 114,780 for standard PSO—a 132x reduction in computational cost.
Conclusion
By combining the global search capabilities of PSO with the predictive power of Gaussian Processes, PSO-GP offers a viable path for optimizing expensive, high-dimensional engineering problems. The dynamic renewal of the training set allows the surrogate to adapt to the landscape in real-time, focusing computational resources on the most promising areas. Future work will focus on the robustness of the algorithm in the presence of noisy simulation data and its scalability to problems involving hundreds of design variables.
Summary updated