Abstract
Many real-world control and classification tasks involve a large number of features, leading to large artificial neural network (ANN) architectures. Neuroevolution (NE) is an effective approach for optimizing ANNs; however, it faces two primary bottlenecks: the high dimensionality of the search space and the time-consuming nature of evaluating large populations on massive datasets. This paper proposes the Limited Evaluation Cooperative Co-evolutionary Differential Evolution (LECCDE) algorithm to address these challenges. LECCDE optimizes the pre-synaptic weights of each post-synaptic neuron in separate subpopulations using a Cooperative Co-evolutionary framework and employs a limited evaluation scheme with fitness inheritance to reduce computational costs. Experimental results across three datasets demonstrate that LECCDE significantly improves accuracy and reduces computing time compared to standard Differential Evolution.
Introduction
Neuroevolution (NE) serves as a powerful alternative to gradient-based methods, particularly in environments where gradients are noisy or unavailable. However, direct-encoding NE—where every network weight is represented as a gene—struggles to scale to modern, high-dimensional neural networks. The two main hurdles are the massive parameter spaces that standard evolutionary algorithms fail to search efficiently and the prohibitive computational cost of full-dataset fitness evaluations. While many researchers pivot to indirect encoding, this work tackles the direct encoding scalability problem by combining divide-and-conquer strategies with efficient data sampling.
Methodology
The LECCDE algorithm enhances standard Differential Evolution (DE) through two key mechanisms: Cooperative Co-evolution (CC) and Limited Evaluation (LE).
Cooperative Co-evolution (CC)
To handle high dimensionality, LECCDE uses a biological heuristic to decompose the network. It assigns the pre-synaptic weights of each post-synaptic neuron to a distinct subpopulation. These subpopulations evolve their specific weights independently but cooperate by combining their best individuals to form a complete global network for fitness evaluation. This structural decomposition, as shown below, partitions a massive weight matrix into logically grouped, manageable chunks.
Limited Evaluation (LE)
To address the computational bottleneck, the LE scheme borrows the concept of mini-batches from stochastic gradient descent. Instead of evaluating networks on the entire dataset every generation, LE evaluates them on small random batches. To prevent noisy evaluations from destabilizing the evolutionary process, it employs "fitness inheritance," blending a network's current batch score with the historical fitness inherited from its parent.
Experimental Results
The authors evaluated LECCDE on standard classification benchmarks: Wisconsin Breast Cancer (WBC), Epileptic Seizure Recognition (ESR), and Human Activity Recognition (HAR). These datasets were chosen to demonstrate how the algorithm scales as feature and instance counts grow.
Accuracy and Convergence
The synergy of CC and LE allows the algorithm to handle both spatial complexity (many weights) and temporal complexity (many training examples). As illustrated in the validation accuracy curves, LECCDE significantly outperforms standard DE and its individual components (LEDE and CCDE). On the HAR dataset, the combined approach achieved a 15-20% accuracy boost over standard DE.
Computational Efficiency
The LE scheme provided massive speedups without degrading accuracy. On the HAR dataset, LECCDE ran approximately 4x faster than standard DE. In extreme scaling tests on MNIST, the impact was even more pronounced, with LECCDE running roughly 25x faster than standard DE while achieving 90.8% test accuracy.
Conclusion and Future Work
LECCDE demonstrates that direct-encoded neuroevolution is viable for large-scale networks when combined with intelligent decomposition and efficient evaluation schemes. While the current implementation uses fixed topologies, future work will explore extending this framework to topology optimization and implementing self-adaptive parameter control to further close the performance gap with state-of-the-art gradient-based methods.
Summary updated