Growing Artificial Neural Networks for Control: the Role of Neuronal Diversity

Eleni Nisioti, Erwan Plantec, Milton Montero, Joachim Winther Pedersen, Sebastian Risi

Introduction

Artificial Intelligence typically relies on fixed neural architectures where only synaptic weights are optimized. In contrast, biological brains grow from a small number of progenitor cells through complex developmental processes. This research explores "growing" Artificial Neural Networks (ANNs) for Reinforcement Learning (RL) tasks, specifically addressing the diversity paradox: the tendency of cells in a shared environment to differentiate into identical types, resulting in useless, homogeneous networks.

Methods

The authors propose Neural Developmental Programs (NDPs), modeling growth as a stochastic graph generation process. Every cell shares three primary sub-models:

  1. DiffModel (Differentiation): A Graph Attention Network (GAT) that updates a cell's state based on its neighbors.
  2. GenModel (Growth): A feedforward network determining if a cell should "bud" to create a new daughter cell.
  3. EdgeModel (Synapse Update): A network that determines the weights of connections between neurons.

To ensure structural complexity, two biological mechanisms are introduced: Intrinsic States (unique IDs for initial cells to create distinct lineages) and Lateral Inhibition (signals that temporarily prevent neighbors from differentiating or growing simultaneously).

The Neural Developmental Program (NDP) growth process and the role of lateral inhibition in maintaining network heterogeneity.

Experimental Setup

The models were evaluated on continuous control benchmarks from the Mujoco Brax library, including Reacher (arm targeting), Inverted Double Pendulum (balancing), and complex locomotion tasks like Halfcheetah and Ant.

Results and Discussion

The study reveals that neuronal diversity is a functional requirement for control; without intrinsic states, NDPs failed to solve any tasks. Lateral inhibition proved crucial as a spatio-temporal regulator, preventing over-growth and maintaining high neuronal diversity throughout the development process.

Neuronal diversity over growth steps, demonstrating how lateral inhibition prevents the collapse into homogeneity.

NDPs achieved performance competitive with established indirect encodings like Hypernetworks across all Brax tasks. While direct encoding (manual weight optimization) reached high scores faster in complex locomotion tasks like the Ant, NDPs offer a significantly smaller search space because the growth program's size is independent of the final network's scale. This "genomic compression" suggests potential for scaling to even larger, more complex controllers.

Conclusion

While growth currently presents a training challenge rather than a clear speed advantage, the introduction of biological regulators like lateral inhibition makes developmental encodings viable for complex RL. Future work aims to integrate environmental feedback during growth, mimicking how biological development continues alongside learning to unlock the full potential of adaptive, growing architectures.

Summary updated

BibTeX
@inproceedings{nisioti2024growing,
  author = {Eleni Nisioti and Erwan Plantec and Milton Montero and Joachim Winther Pedersen and Sebastian Risi},
  title = {{Growing Artificial Neural Networks for Control: the Role of Neuronal Diversity}},
  booktitle = {GECCO Companion},
  year = {2024},
  doi = {10.1145/3638530.3654356},
}
Figure