A self-organising neural network for modelling cortical development

Michael W. Spratling, Gillian Hayes

Abstract

This paper introduces a biologically inspired self-organizing neural network designed to model the development of structured topological maps in the brain's cortex. Departing from traditional models like the Kohonen Self-Organizing Map (SOM), the proposed architecture utilizes synaptogenesis from zero, covariance-based learning, and population coding. These mechanisms allow for a more realistic representation of cortical development, where connections grow based on activity rather than starting from random initializations. Experimental results demonstrate the model's ability to form ordered 2D maps, handle discontinuous input spaces, and replicate orientation preference columns.

Introduction

The brain's cortex is organized into topological maps where neurons in close proximity respond to similar sensory stimuli. Computational modeling of this organization has largely relied on the Kohonen Self-Organizing Map. However, the SOM incorporates several biologically implausible features: random weight initialization, artificial weight normalization to prevent runaway activity, and a "winner-take-all" activation scheme that ignores the distributed nature of neural firing. This work aims to bridge the gap between abstract algorithms and biological reality by introducing a model where synapses grow from an initial state of zero and information is represented through distributed population coding.

Methodology

The proposed algorithm introduces four primary innovations to enhance biological plausibility:

  1. Synaptogenesis from Zero: All synaptic weights ($q_{ij}$) are initialized at zero. Learning involves the growth of new connections rather than the reorganization of random ones. This ensures the network's output is "inaccurate rather than wrong" during early learning.
  2. Covariance-based Learning: To avoid hard normalization, a variant of the covariance rule is employed. Moving averages of past activity for both inputs and outputs are tracked; synapses strengthen when activity exceeds the average and weaken when it falls below, leading to natural weight stabilization.
  3. Habituation: A "conscience" mechanism tracks node winning frequency. Nodes that win too often have their competitiveness temporarily reduced, ensuring the entire input space is represented across the network.
  4. Population Coding: Weak lateral inhibition allows a neighborhood of neurons around a winner to remain active. This distributed output can serve directly as input for subsequent layers, enabling modular hierarchies.

Results and Evaluation

The model was evaluated using synthetic datasets designed to test topological mapping, including a unit square, discontinuous quadrants (to test map "stretching"), and orientation data based on the von der Malsburg (1973) model.

Figure 1: Mapping results for (a) unit square, (b) discontinuous quadrants, and (c) orientation columns.

The network successfully formed ordered 2D maps from the unit square input and adapted to discontinuous inputs by stretching the map across gaps (Figure 1a, b). Furthermore, it replicated orientation preference maps characteristic of the primary visual cortex (Figure 1c).

Figure 2: Development of the topological map over 4,000 iterations, starting from zero synaptic weights.

The "zero-start" approach demonstrated significant robustness, reliably converging to a stable, topologically organized state over 4,000 iterations (Figure 2). While biologically more plausible, the authors acknowledge a trade-off in efficiency: the model requires approximately ten times more training data than the Kohonen algorithm to achieve comparable results.

Discussion and Conclusion

By starting from zero weights and utilizing activity-dependent learning rules, the model demonstrates that competition can be achieved without mathematical "hacks" like hard weight normalization. The use of population coding for both inputs and outputs suggests a modular architecture capable of modeling complex cortical hierarchies. Future work will involve linking multiple networks together to see if the hierarchy maintains its self-organizing properties and exploring the parameter sensitivity of the synaptogenesis process.

Summary updated

Abstract

This paper presents a novel self-organising neural network. It has been developed for use as a simplified model of cortical development. Unlike many other models of topological map formation all synaptic weights start at\nzero strength (so that synaptogenesis might be modelled). In addition, the algorithm works with the same format of encoding for both inputs to and outputs from the network (so that the transfer and recoding of information between cortical regions might be modelled).
BibTeX
@inproceedings{spratling1998self,
  author = {Michael W. Spratling and Gillian Hayes},
  title = {{A self-organising neural network for modelling cortical development}},
  booktitle = {ESANN},
  year = {1998},
  url = {https://openreview.net/forum?id=Xu2fcHT_r44},
}
Figure