In traditional deep learning, we typically optimize a single model by following the steepest descent along a loss landscape. While efficient, this gradient-based approach is prone to getting stuck in local optima and requires tedious manual tuning of hyperparameters. Population-based training (PBT) and evolutionary algorithms (EA) offer a powerful alternative: instead of one model, they evolve a diverse "population" of candidates, selecting and mutating the best performers to discover superior architectures, schedules, and weights.
However, this robustness comes with a significant tax. The primary challenge of these methods is their immense computational overhead, which often scales linearly with the size of the population.
The Linear Scaling of Resource Demand
The most intuitive reason for the high cost of PBT is simple arithmetic. If training a single neural network requires a certain amount of GPU memory and time, training a population of $N$ networks simultaneously requires roughly $N$ times those resources. Cui et al. note that while parallelization can keep "wall-clock" time manageable, the aggregate GPU hours scale linearly with population size.
This resource intensity creates a high barrier to entry. For example, Cheng et al. and Leng et al. demonstrate that using PBT to automate data augmentation policies for 3D object detection yields state-of-the-art results, but necessitates massive hardware infrastructure that is often inaccessible to smaller research teams. Similarly, Zhu et al. highlight that self-tuning Graph Convolutional Networks (GCNs) requires significant memory and parallel processing capabilities to evolve multiple model instances in real-time.
Fitness Evaluation as a Bottleneck
In a standard genetic algorithm, "fitness" might be a simple mathematical function. In deep learning, the fitness of an individual is usually its performance on a validation set after some amount of training. This means that every generation of the evolutionary loop requires a full evaluation cycle for every member of the population.
As Narita et al. point out, calculating the error for every individual in a population during each generation makes the per-step resource requirements significantly higher than gradient-based optimization. This is particularly taxing in tasks like selective rationalization, where Ruggeri et al. observe that training multiple candidate models from scratch in each generation prevents the rapid iteration cycles common in gradient-based research. Even in hybrid models like the EA-LSTM, Li et al. explain that evaluating the fitness of an entire population via multiple forward passes is inherently more resource-intensive than a single backward pass.
The Reinforcement Learning Multiplier
The overhead becomes even more pronounced in Reinforcement Learning (RL). To evaluate the "fitness" of a reward function or a policy, one must often train an agent from scratch within a simulation.
- Reward Design: Frameworks like REvolve and LaRes use Large Language Models to evolve reward functions. Hazra et al. and Li et al. both emphasize that training multiple RL policies in parallel for every generation requires massive GPU clusters.
- Multi-Agent Systems: In competitive settings, the complexity grows. Ernst notes that population-based training requires significantly more environment interactions than simple self-play. When training against a diverse population of adversaries to ensure robustness, Liang et al. argue that the need to train multiple generations of both agents and adversaries leads to significantly longer training durations.
Why We Pay the Price
If the costs are so high, why do researchers continue to use these methods? The answer lies in the limitations of gradients. Gradient descent is a local search; it cannot easily navigate landscapes with many local minima or discrete hyperparameter choices.
Hasegawa demonstrates that self-adaptive evolutionary strategies can outperform traditional methods in precision and convergence speed by avoiding manual tuning. In the context of Generative Adversarial Networks (GANs), Wang et al. show that evolving a population of generators with different objectives effectively eliminates mode collapse, a problem that often plagues single-model GAN training.
The research direction, therefore, is not just about using populations, but about finding ways to mitigate their cost. Whether through "kickstarting" students with teachers to accelerate learning (Schmitt et al.) or using "fictitious cross-play" to bridge the gap between self-play efficiency and population robustness (Wu et al.), the goal is to harness the global search power of populations without the prohibitive computational tax.
Key Foundational Work
Nuclear Reactor Diagnostic System Using GA Trained Neural Networks (1994) — contributed 1 concept elements to this direction.
Training Artificial Neural Network Using Modification of Differential Evolution Algorithm (2015) — contributed 1 concept elements to this direction.