Abstract
This paper introduces a biologically inspired self-organizing neural network designed to model the development of structured topological maps in the brain's cortex. Departing from traditional models like the Kohonen Self-Organizing Map (SOM), the proposed architecture utilizes synaptogenesis from zero, covariance-based learning, and population coding. These mechanisms allow for a more realistic representation of cortical development, where connections grow based on activity rather than starting from random initializations. Experimental results demonstrate the model's ability to form ordered 2D maps, handle discontinuous input spaces, and replicate orientation preference columns.
Introduction
The brain's cortex is organized into topological maps where neurons in close proximity respond to similar sensory stimuli. Computational modeling of this organization has largely relied on the Kohonen Self-Organizing Map. However, the SOM incorporates several biologically implausible features: random weight initialization, artificial weight normalization to prevent runaway activity, and a "winner-take-all" activation scheme that ignores the distributed nature of neural firing. This work aims to bridge the gap between abstract algorithms and biological reality by introducing a model where synapses grow from an initial state of zero and information is represented through distributed population coding.
Methodology
The proposed algorithm introduces four primary innovations to enhance biological plausibility:
- Synaptogenesis from Zero: All synaptic weights ($q_{ij}$) are initialized at zero. Learning involves the growth of new connections rather than the reorganization of random ones. This ensures the network's output is "inaccurate rather than wrong" during early learning.
- Covariance-based Learning: To avoid hard normalization, a variant of the covariance rule is employed. Moving averages of past activity for both inputs and outputs are tracked; synapses strengthen when activity exceeds the average and weaken when it falls below, leading to natural weight stabilization.
- Habituation: A "conscience" mechanism tracks node winning frequency. Nodes that win too often have their competitiveness temporarily reduced, ensuring the entire input space is represented across the network.
- Population Coding: Weak lateral inhibition allows a neighborhood of neurons around a winner to remain active. This distributed output can serve directly as input for subsequent layers, enabling modular hierarchies.
Results and Evaluation
The model was evaluated using synthetic datasets designed to test topological mapping, including a unit square, discontinuous quadrants (to test map "stretching"), and orientation data based on the von der Malsburg (1973) model.
The network successfully formed ordered 2D maps from the unit square input and adapted to discontinuous inputs by stretching the map across gaps (Figure 1a, b). Furthermore, it replicated orientation preference maps characteristic of the primary visual cortex (Figure 1c).
The "zero-start" approach demonstrated significant robustness, reliably converging to a stable, topologically organized state over 4,000 iterations (Figure 2). While biologically more plausible, the authors acknowledge a trade-off in efficiency: the model requires approximately ten times more training data than the Kohonen algorithm to achieve comparable results.
Discussion and Conclusion
By starting from zero weights and utilizing activity-dependent learning rules, the model demonstrates that competition can be achieved without mathematical "hacks" like hard weight normalization. The use of population coding for both inputs and outputs suggests a modular architecture capable of modeling complex cortical hierarchies. Future work will involve linking multiple networks together to see if the hierarchy maintains its self-organizing properties and exploring the parameter sensitivity of the synaptogenesis process.
Summary updated