Introduction
Artificial Intelligence typically relies on fixed neural architectures where only synaptic weights are optimized. In contrast, biological brains grow from a small number of progenitor cells through complex developmental processes. This research explores "growing" Artificial Neural Networks (ANNs) for Reinforcement Learning (RL) tasks, specifically addressing the diversity paradox: the tendency of cells in a shared environment to differentiate into identical types, resulting in useless, homogeneous networks.
Methods
The authors propose Neural Developmental Programs (NDPs), modeling growth as a stochastic graph generation process. Every cell shares three primary sub-models:
- DiffModel (Differentiation): A Graph Attention Network (GAT) that updates a cell's state based on its neighbors.
- GenModel (Growth): A feedforward network determining if a cell should "bud" to create a new daughter cell.
- EdgeModel (Synapse Update): A network that determines the weights of connections between neurons.
To ensure structural complexity, two biological mechanisms are introduced: Intrinsic States (unique IDs for initial cells to create distinct lineages) and Lateral Inhibition (signals that temporarily prevent neighbors from differentiating or growing simultaneously).
Experimental Setup
The models were evaluated on continuous control benchmarks from the Mujoco Brax library, including Reacher (arm targeting), Inverted Double Pendulum (balancing), and complex locomotion tasks like Halfcheetah and Ant.
Results and Discussion
The study reveals that neuronal diversity is a functional requirement for control; without intrinsic states, NDPs failed to solve any tasks. Lateral inhibition proved crucial as a spatio-temporal regulator, preventing over-growth and maintaining high neuronal diversity throughout the development process.
NDPs achieved performance competitive with established indirect encodings like Hypernetworks across all Brax tasks. While direct encoding (manual weight optimization) reached high scores faster in complex locomotion tasks like the Ant, NDPs offer a significantly smaller search space because the growth program's size is independent of the final network's scale. This "genomic compression" suggests potential for scaling to even larger, more complex controllers.
Conclusion
While growth currently presents a training challenge rather than a clear speed advantage, the introduction of biological regulators like lateral inhibition makes developmental encodings viable for complex RL. Future work aims to integrate environmental feedback during growth, mimicking how biological development continues alongside learning to unlock the full potential of adaptive, growing architectures.
Summary updated