Abstract
Artificial spike-based computation, inspired by models of computation in the central nervous system, may present significant performance advantages over traditional methods for specific types of large scale problems. This paper describes very simple network architectures for k-winners-take-all and soft-winner-take-all computation using neural oscillators. Fast convergence is achieved from arbitrary initial conditions, which makes the networks particularly suitable to track time-varying inputs.
Introduction
In neural computation, "Winner-Take-All" (WTA) is a fundamental primitive where only the neuron with the strongest input remains active while others are suppressed. Its generalizations—$k$-WTA (identifying the top $k$ inputs) and soft-WTA (rank-ordering all inputs)—are essential for tasks like competitive learning, pattern recognition, and selective attention.
Traditional artificial neural network models for $k$-WTA often struggle with convergence speed, sensitivity to initial conditions, and the ability to track rapidly changing inputs. This paper introduces a biologically inspired architecture based on synchronized oscillations that achieves $O(n)$ complexity and converges within just two spiking periods.
Network Architecture and Mechanisms
The proposed network utilizes the FitzHugh-Nagumo (FN) model, a simplified version of the Hodgkin-Huxley equations that describes how neurons generate spikes. The architecture consists of $n$ FN oscillators, each receiving an external input $I_i$, and a single global inhibitory neuron that provides feedback to suppress the network.
The $k$-WTA Mechanism
To select $k$ winners, a local self-inhibition variable $u_i$ is introduced for each neuron. When an FN neuron spikes, its local inhibition charges. The global inhibitory neuron is configured to trigger only after $k$ neurons have spiked by monitoring the sum of these local inhibitions. Once the $k^{th}$ neuron fires, the global inhibitor fires, resetting the entire network for a new cycle.
The Soft-WTA Mechanism
For soft-WTA, the goal is to rank-order all inputs. By setting $k=n$ and introducing a lower bound on the global inhibition $z_{low}$, the inhibition decays slowly enough for every neuron to spike in order of its input strength before the global reset occurs.
Results and Performance
The core insight of this model is temporal rank-ordering: as global inhibition decays, neurons with stronger inputs reach their firing threshold sooner. The timing of spikes effectively encodes the rank of the inputs.
Numerical simulations, such as the 10-neuron network shown in Figure 2, confirm that the system identifies the $k$ largest inputs within two cycles regardless of the starting state. The first period clears arbitrary initial conditions, and the second establishes the correct firing order. This fast convergence allows the network to track time-varying inputs effectively, as demonstrated in experiments with sinusoidal signals.
Conclusion
The proposed oscillatory model offers a scalable ($O(n)$) and robust solution for competitive neural computation. While performance depends on parameter tuning (e.g., discharge rates and thresholds), the architecture's ability to handle dynamic inputs and ties makes it a promising building block for higher-level cognitive models like selective attention, associative memory, and perceptual binding.
Summary updated