This 1988 paper by André J. Noest introduces a framework for neural networks that move beyond simple binary or scalar values, instead using "phasors"—unit-length 2-vectors represented as complex numbers. By encoding information in the phase rather than just magnitude, these networks provide a more natural way to model biological oscillators, optical computers, and circular data like edge orientations. Ultimately, this architecture naturally handles periodic data and provides a mathematically elegant way to build associative memories that are both biologically plausible and optimized for optical hardware.
The Problem with Scalar Neurons
Most traditional neural networks of the era relied on binary states (0/1) or scalars passed through sigmoidal functions. Noest argues that this is "awkward" when dealing with signals that have a circular topology—such as the phase of a sound wave, the direction of local optic flow in an image, or the orientation of an edge.
Furthermore, biological systems (like the olfactory bulb) often use limit-cycle oscillators as building blocks. Standard models struggle to represent these oscillators efficiently. Noest seeks a model where the nonlinearity (clipping the vector to unit length) acts orthogonally to the information being processed (the phase angle), allowing for cleaner linear relations during computation.
The Phasor Framework
The core of the paper is the definition of a network where each neuron $s_i$ is a complex number with a magnitude of 1 ($|s_i| = 1$).
- Complex Couplings: Unlike standard weights, the "synaptic" weights $c_{ij}$ are also complex-valued. This allows the network to represent propagation delays as phase shifts ($f \cdot d \cdot 2\pi$ radians).
- Dynamics: The author defines three types of dynamics: continuous-time parallel evolution (Type A), discrete-time serial updating (Type B), and discrete-time parallel updating (Type C).
- Energy Functions: For Hermitian coupling matrices ($c_{ij} = \bar{c}_{ji}$), Noest derives a Lyapunov (energy) function, proving that the network will naturally relax into stable states (minima).
- Phasor Hebbian Learning: To store patterns, the author adapts Hebb’s rule to the complex domain: $c_{ij} = \sum_k s_i^{(k)} \bar{s}_j^{(k)}$, where $\bar{s}$ is the complex conjugate.
- Analytical Modeling: To find exact solutions for performance, the author utilizes a model with sparse, asymmetric connectivity (where each cell has $z$ incoming connections from $N$ total cells).
Key Insights
- Rotational Invariance: The network’s behavior is invariant under a global rotation. This means the system doesn't care about the absolute phase, only the relative phase between neurons, which mirrors how many biological and physical sensors operate.
- Orthogonal Nonlinearity: In a standard neuron, the nonlinearity (like a sigmoid) squashes the signal itself. In a phasor neuron, the nonlinearity only affects the length of the vector (keeping it on the unit circle), leaving the phase (the actual information) to interact linearly.
- Biological Realism: Complex couplings are a more faithful representation of biological brains because they account for the inevitable propagation delays between neurons.
- Optical Computing Synergy: Phasor networks are a perfect match for optical neurocomputers, where information is already carried in the phase of light beams, potentially avoiding the need for slow optical switching elements.
Performance and Capacity
Because the paper is a theoretical and mathematical derivation, it evaluates performance using randomly generated sets of $p$ patterns consisting of $N$ phasors, rather than real-world datasets. These patterns are assumed to be orthogonal or random in the limit where $N \to \infty$.
The author calculates the maximum storage capacity of the network. For phasor nets, the critical loading capacity is $\alpha_c = p/z = \pi/4 \approx 0.785$. This capacity is notably higher than the $2/\pi \approx 0.637$ found in similar sparse binary networks.
The paper provides an exact formula for retrieval accuracy ($M$) versus the "loading-plus-noise" factor ($a$). As shown in the figure above, perfect recall occurs in a single pass when loading is very low, and the fixed point $M$ remains viable until the critical capacity threshold is reached. Additionally, the author derives the distribution of "phase scatter," showing how noise and loading cause the retrieved phasors to deviate from their ideal stored angles.
Future Directions
Noest outlines several avenues for future exploration:
- Symmetry Breaking: The current model assumes a perfectly circular symmetry ($O(2)$). "Breaking" this symmetry—for example, by forcing neurons into discrete positions on the circle—could lead to interesting new behaviors.
- Higher Dimensions: While focused on 2D vectors (circles), the underlying mathematics could be generalized to $k$-spheres (higher-dimensional unit vectors).
- Algorithm Generalization: Beyond associative memory, other popular algorithms like backpropagation could be generalized to the phasor setting.
- Hardware Implementation: These models hold strong potential for physical implementations in Josephson-junctions or phase-encoded optics, optimizing functions of angular distance.
Summary updated