Attractor Dynamics in Asymmetric Recurrent Networks

31 concepts23 papers5 proposals

In classical neural network theory, memory retrieval is often visualized as a ball rolling down a hill. A network is initialized with a noisy input, and its state evolves over time until it settles into the lowest point of a nearby valley. This valley is a stable fixed point—a stored memory.

Mathematically, this elegant behavior relies on a strict assumption: the connection matrix must be perfectly symmetric ($W_{ij} = W_{ji}$). As Michael Fleisher notes in The Hopfield Model with Multi-Level Neurons, traditional proofs of network convergence depend entirely on this lack of directional bias. Symmetry guarantees the existence of a scalar energy function (a Lyapunov function) that strictly decreases with every state update, preventing the network from oscillating or behaving chaotically.

However, biological brains completely reject this constraint. Synapses are fundamentally directional; a neuron can excite its neighbor without receiving an equal and opposite reaction. When we break symmetry, the clean energy landscape vanishes. Instead of a conservative gravitational field, the network's state space becomes a swirling fluid dynamics problem where states can loop, diverge, or drift endlessly.

How can a recurrent network perform reliable computation or store stable memories when it lacks an energy landscape?

The Consequence of Breaking Symmetry

When connection weights are asymmetric, the network's updates can no longer be written as the gradient of a single global energy function. Without this mathematical anchor, the system can easily become unstable.

As Ali A. Minai et al. demonstrated in Predicting Complex Behavior in Sparse Asymmetric Networks, asymmetric recurrent networks frequently exhibit highly unpredictable global dynamics, fluctuating erratically or getting trapped in repetitive loops. Furthermore, in continuous attractor networks—which are used to model working memory—unregulated asymmetries can destroy the stability of stored representations. Alexander Seeholzer et al. showed in Stability of working memory in continuous attractor networks under the control of short-term plasticity that systematic asymmetries introduce a directional bias that causes stored continuous memories to rotate or drift over time rather than remaining stationary.

Yet, despite these theoretical hazards, biological networks successfully maintain stable perceptions and retrieve memories. This paradox has driven researchers to find alternative mathematical frameworks that explain why asymmetric networks can still converge to stable attractors.

Taming Asymmetry Without Energy Landscapes

To prove that an asymmetric network can settle into stable fixed points, researchers have moved away from global energy minimization toward more general dynamical systems principles.

1. General Vector Fields and Modified Lyapunov Functions

Instead of forcing a network to conform to a scalar energy function, the system can be modeled as a general vector field. In this view, a stable fixed point is not a valley floor, but a point where all the directional forces in the vector field perfectly balance out to zero.

In Extending the Framework of Equilibrium Propagation to General Dynamics, Benjamin Scellier et al. demonstrated that recurrent networks can settle into stable fixed points using general vector fields that completely bypass symmetric weight constraints. To prove this stability rigorously, authors have developed new classes of Lyapunov functions. For example, Fanaskov Vladimir et al. in Associative memory and dead neurons formulated modified dynamical systems that guarantee formal stability even when networks feature highly asymmetric feedforward and feedback paths. Similarly, Anthony N. Michel et al. in Stability Results for Neural Networks applied large-scale dynamical systems theory to show that asymptotic and exponential stability can be achieved even when weight symmetry cannot be guaranteed.

2. Excitatory-Inhibitory Balance

In biological systems, asymmetry is often highly structured, characterized by distinct populations of excitatory and inhibitory neurons. This structure can be leveraged to guarantee stability.

H. Sebastian Seung et al. addressed this in Minimax and Hamiltonian Dynamics of Excitatory-Inhibitory Networks, showing that while traditional energy landscapes fail for non-reciprocal connections, stability can be proven by framing the network as a dissipative Hamiltonian system undergoing minimax optimization. From a different angle, Eli Shlizerman et al. in Modeling the dynamics of neural codes in the olfaction of the Manduca-sexta moth proved that by projecting dynamical equations onto orthogonal spatial patterns, networks can support unique and robust fixed points without requiring symmetric connectivity between individual units.

3. Local Consistency Over Global Minimization

Another way to achieve stability without global symmetry is to focus on local constraints. In An asymmetric associative memory model based on relaxation labeling processes, Marcello Pelillo et al. proposed an associative memory model based on a compatibility matrix that does not require symmetry. Instead of minimizing a global energy function, the network relies on local consistency; strictly consistent states naturally act as local attractors that the system gravitates toward during relaxation, successfully preserving pattern convergence without reciprocal weights.

Symmetric Networks (Hopfield)         Asymmetric Networks (Vector Fields)
       [ Energy Landscape ]                  [ Balanced Vector Field ]
               \                                        |
                \  (Gradient Descent)                   v  (Balanced Forces)
                 v                                   =======> . <=======
           Stable Minimum                         Stable Fixed Point

The Computational Payoff of Asymmetry

If symmetric networks are so much easier to analyze, why build asymmetric ones? The answer lies in computational expressivity and capacity. Symmetric networks are fundamentally constrained by thermodynamic equilibrium and detailed balance, which limits the types of distributions they can model.

Breaking Detailed Balance

In physical terms, a symmetric network is a closed system at equilibrium. Emanuele Panizon in Machine learning at the mesoscale: a computation-dissipation bottleneck demonstrates that breaking the symmetry of interactions (non-reciprocity) allows a network to break equilibrium. This non-equilibrium state unlocks significantly higher expressive capacity for non-linear classification tasks, though it requires the system to dissipate energy.

By abandoning detailed balance, asymmetric networks can also model complex, time-dependent trajectories rather than just static points. Anirudh Goyal et al. leveraged this in Variational Walkback: Learning a Transition Operator as a Stochastic Recurrent Net, showing that training transition operators without the constraint of detailed balance allows asymmetric recurrent networks to sculpt highly complex, non-equilibrium data distributions.

Sequence Storage and Robustness

Asymmetry also fundamentally changes memory capacity. Because symmetric networks must settle into static points, they struggle to store sequences of states.

In Phase Diagram and Storage Capacity of Sequence-Storing Neural Networks, A. Düring et al. used dynamical path integral methods to analyze networks that evolve through sequences of states. They discovered that these asymmetric, sequence-storing networks are nearly twice as efficient as standard Hopfield models, achieving a significantly higher critical storage capacity.

Furthermore, this functional memory retrieval is remarkably robust. Anna Levina et al. in Critical dynamics in associative memory networks demonstrated that associative memory retrieval is highly resilient to synaptic asymmetry introduced by homeostatic regulatory processes. Similarly, Leo Kozachkov et al. in Neuron-Astrocyte Associative Memory showed that high-capacity associative memory systems remain fully functional even when strict mathematical symmetries are relaxed, proving that stable attractor dynamics do not require the rigid, idealized constraints of traditional energy-based models.

Key Foundational Work

Stability Results for Neural Networks (1987) — contributed 2 concept elements to this direction.
The Hopfield Model with Multi-Level Neurons (1987) — contributed 1 concept elements to this direction.
Predicting Complex Behavior in Sparse Asymmetric Networks (1992) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_non_symmetric_attractor_dynamics_136ab75e,
  title = {non-symmetric attractor dynamics},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/non-symmetric-attractor-dynamics/rdr_136ab75e27bf4703b5e8a5092c8f60d5},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-22},
}

This exact version (version 1, frozen text):

@misc{lacuna_non_symmetric_attractor_dynamics_136ab75e_v1,
  title = {non-symmetric attractor dynamics},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/non-symmetric-attractor-dynamics/rdr_136ab75e27bf4703b5e8a5092c8f60d5/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-22},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake