Abstract
Adapting to task changes without forgetting previous knowledge is a key skill for intelligent systems, and a crucial aspect of lifelong learning. Swarm controllers, however, are typically designed for specific tasks, lacking the ability to retain knowledge across changing tasks. We introduce a lifelong evolutionary framework for swarms, where a population of swarm controllers is evolved in a dynamic environment that incrementally presents novel tasks. We discover that the population inherently preserves information about previous tasks, while the top-performing individual for a given task catastrophically forgets. To mitigate this, we design a regularization process for the evolutionary algorithm, reducing forgetting in top-performing individuals.
Introduction
In swarm robotics, controllers are typically evolved for static, specific goals. However, real-world environments are dynamic, requiring swarms to adapt to shifting objectives. When a swarm's task changes—for instance, from foraging red items to green ones—the evolutionary process often optimizes for the new task at the expense of the old, a phenomenon known as catastrophic forgetting.
While lifelong learning has been extensively studied for individual agents, its application to the emergent behaviors of swarms remains largely unexplored. This paper introduces a lifelong evolutionary framework designed to enable swarms to adapt to novel tasks incrementally while retaining knowledge of previous ones, ensuring the collective remains robust in changing environments.
Methodology: The Lifelong Evolutionary Framework
The proposed framework utilizes NeuroEvolution of Augmenting Topologies (NEAT) to evolve both the weights and structures of neural network controllers for swarm agents.
The system operates in a simulated environment, SwarmForagingEnv, where robots must identify and deliver specific color-coded boxes. The core of the approach involves:
- Dynamic Task Drifts: Subjecting the swarm to a sequence of tasks (e.g., Task A → Task B → Task A) to evaluate adaptation and memory.
- Genetic Distance Regularization: To mitigate forgetting in top-performing individuals, a penalty is added to the fitness function. This penalty, inspired by Elastic Weight Consolidation (EWC) in deep learning, discourages controllers from deviating too far genetically from successful ancestors of previous tasks, using NEAT's inherent genetic distance metric.
Key Insights and Results
The research reveals a fundamental distinction between population-level and individual-level memory.
The Population as a Collective Buffer
A significant finding is that the evolved population naturally resists forgetting better than any single agent. Due to the speciation inherent in NEAT, the "gene pool" maintains a diverse set of solutions. Even when the current "best" individual has specialized for a new task, other members of the population often retain the capacity to solve previous tasks. This collective diversity allows the swarm to re-adapt to old tasks up to four times faster than the initial learning phase.
Individual Forgetting and Regularization
Despite the population's robustness, the top-performing individuals typically suffer from total amnesia regarding prior tasks without intervention.
By implementing genetic distance regularization, the researchers significantly reduced this forgetting. In experiments, the fitness loss on old tasks was reduced from approximately 22.5 points to just 5.0 points. However, this introduces a stability-plasticity trade-off: high regularization preserves memory but can hinder the learning of new, complex behaviors. The authors found a "sweet spot" that allows for both rapid adaptation and significant retention.
Conclusion
This work demonstrates that "collective memory" is an emergent property of evolutionary swarms. While the population acts as a natural buffer against forgetting, maintaining high performance in individual leaders requires targeted algorithmic interventions like genetic regularization. Future research will explore how to maintain genetic diversity while regularizing, and how these frameworks scale to larger swarms and more diverse task sets beyond foraging.
Summary updated