Evolution and Learning in Neural Networks: The Number and Distribution of Learning Trials Affect the Rate of Evolution

NeurIPS 1990

TL;DR

This paper explores how learning impacts the evolutionary process in artificial neural networks. They found that an intermediate amount of learning, distributed unevenly among individuals, optimizes the rate of genetic evolution, leading to faster adaptation than either no learning or excessive learning. This suggests a powerful synergy between learning and evolution for tackling complex problems.

What Problem Does This Paper Address?

The paper investigates the interplay between evolution and learning in artificial neural networks, specifically how these two processes influence each other's speed and effectiveness. While evolution (driven by genetic algorithms) is good at exploring complex solution spaces and avoiding local optima, it can be slow. Learning, on the other hand, can quickly refine an individual's performance. The question is: how can these two mechanisms be combined to accelerate the overall evolutionary process? The authors draw inspiration from the Baldwin effect in biology, which posits that learned behaviors can indirectly influence genetic evolution. They aim to demonstrate this effect in artificial neural networks and understand how the amount and distribution of learning trials impact the rate of genetic change.

Methods

The researchers developed a hybrid system that combines a genetic algorithm (GA) with gradient-descent learning (specifically, perceptron learning).

  • Genetic Algorithm (GA): This component handles the evolutionary aspect. It maintains a population of neural networks, where each network's "genes" encode its initial connection weights. The GA uses fitness-proportional reproduction, mutation, and crossover to evolve these genes over generations.
  • Gradient-Descent Learning: Before selection and reproduction, each network in the population undergoes training on a pattern recognition task. This learning process modifies the network's weights to improve its performance.
  • Fitness Evaluation: The "fitness" of a network is determined by its ability to correctly classify patterns after undergoing learning. This fitness score then dictates its reproductive success in the GA.
  • Learning Schemes: Two primary learning distribution schemes were simulated:
  • Uniform Learning: Every network in a generation receives the same number of training presentations.
  • Distributed Learning: Networks receive a randomly chosen number of training presentations, leading to variation in learning experience within a generation.

Datasets

The paper uses a custom dataset for a pattern recognition task: classifying pixel images of the first five letters of the alphabet. The input is a 9x9 pixel grid, processed by orientation-selective filters, and the output is one of five category units. No standard benchmark datasets are mentioned.

Key Insights and Ideas

  • The Baldwin Effect in Artificial Neural Networks: Learning can indeed accelerate genetic evolution. Networks that learn well, even if their initial "genes" aren't perfect, can be selected, and their "better" genes can be passed on, leading to a population with superior genetic predispositions over time.
  • The "Sweet Spot" for Learning: Too little learning means the GA has to do all the work, making evolution slow. Too much learning, however, can also hinder evolution. If all networks become perfect after learning, selection becomes random, as there's no difference in performance to guide the selection of "better" genes. An intermediate amount of learning is optimal.
  • Distribution Matters: Unevenly distributing learning trials (distributed learning) leads to faster evolution than giving everyone the same amount (uniform learning). This is because it maintains "pressure" on the genes to improve, as some individuals might not learn perfectly and thus rely more on their genetic makeup for survival.
  • Learning vs. Genes: A network's performance after learning is more dependent on its ancestors' learned experiences than on its own immediate learning. This highlights the indirect but powerful influence of learning on genetic evolution.

Key Results

  • Learning Accelerates Evolution: Both uniform and distributed learning schemes showed significantly faster evolution (higher population average fitness at birth) compared to no learning.
  • Optimal Learning Amount: Figure 5 demonstrates that an intermediate amount of learning (around 10-100 trials per individual) leads to the highest population fitness at generation 100. Excessive learning (e.g., 500 presentations) resulted in the slowest evolution.
  • Distributed Learning Advantage: The distributed learning scheme resulted in significantly faster evolution than uniform learning, performing up to four times better than a pure GA alone.
  • Small Learning Improvement, Big Evolutionary Impact: The fitness improvement after learning in a single generation was typically small (around 5%), yet this small improvement was sufficient to drive substantial genetic evolution over many generations.

Limitations and Future Work

  • Limitations:
    • The study focuses on a single-minimum landscape given by perceptron learning. The advantages of the hybrid approach might be even more pronounced in landscapes with multiple minima.
  • The pattern recognition task is relatively simple (first five letters of the alphabet).
  • Future Work: The authors suggest that the advantages of their hybrid approach could be even more significant in more complex, high-dimensional pattern recognition problems with multiple local minima, where evolutionary search alone can struggle. They also hint at the potential for this approach in other complex problems.

Summary updated

BibTeX
@inproceedings{1990evolution,
  title = {{Evolution and Learning in Neural Networks: The Number and Distribution of Learning Trials Affect the Rate of Evolution}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {1990},
}

Loading related content...

Figure