Abstract
This paper introduces Spiking Weight Agnostic Neural Networks (Spiking WANNs), a method for evolving neural network topologies that function effectively using discrete, "spiking" signals without requiring traditional weight training. By restricting architecture search to simple threshold and linear activation functions, the researchers evolved remarkably sparse networks that are inherently compatible with low-power neuromorphic hardware.
Introduction
Traditional Artificial Neural Networks (ANNs) are energy-intensive, making them difficult to deploy on "edge" devices like drones or satellites. Neuromorphic computing offers a solution by mimicking the brain's energy efficiency through Spiking Neural Networks (SNNs), which communicate via discrete pulses (spikes) rather than continuous values. However, two major hurdles remain: training difficulty and the high computational cost of Neural Architecture Search (NAS). This research addresses these by asking: Can we evolve network structures that are "born" ready to spike and perform tasks well regardless of their specific weight values?
Methods
The research builds on the Weight Agnostic Neural Network (WANN) framework, where the goal is to find a topology that performs a task even when all its connections share a single, random weight value. To ensure compatibility with neuromorphic hardware, the authors restricted the activation functions to two types: Threshold (Step) Functions, mimicking biological neuron firing, and Linear Functions, mimicking additive dendritic trees. The search uses an evolutionary algorithm that starts with a minimal network and iteratively adds or removes connections, optimizing for performance across a range of shared weight values and overall simplicity.
Results
The approach was evaluated on three benchmark tasks: Swingup Cartpole, Bipedal Walker, and MNIST (downsampled to 16x16 pixels). Spiking WANNs consistently evolved smaller, sparser networks than standard WANNs. For example, in the Bipedal Walker task, the spiking version used approximately 38% fewer connections while maintaining comparable performance.
Key findings include:
- Simplicity is Sufficient: Simple threshold and linear gates are enough to evolve functional "circuit motifs."
- Extreme Sparsity: Spiking WANNs are significantly more compact. On MNIST, the Spiking WANN achieved 87.7% accuracy using less than half the connections of the standard WANN (576 vs 1228).
- Control Task Performance: In control tasks, Spiking WANNs outperformed original WANNs in "shared weight" scenarios. For Swingup Cartpole, Spiking WANNs achieved a score of 745 ± 11 compared to 723 ± 16 for standard WANNs.
- Hardware Readiness: These networks are "spiking-ready" and can be mapped onto neuromorphic chips like Intel’s Loihi or IBM’s TrueNorth far more easily than traditional models.
Discussion and Future Work
While Spiking WANNs show promise, they do not yet fully utilize the temporal dynamics common in Leaky Integrate-and-Fire (LIF) models. The performance drop in MNIST suggests that classification tasks may require more complex topologies or different optimization strategies than control tasks. Future directions include incorporating energy-consumption metrics directly into the evolutionary fitness function and testing resilience to synaptic noise common in analog hardware.
Conclusion
Spiking WANNs demonstrate that effective spiking architectures can be evolved without weight training, leading to extremely sparse and hardware-compatible models. This approach paves the way for efficient, topology-driven design of neuromorphic systems.
Summary updated