Beyond Fixed Points: Superpolynomial Capacity of Asymmetric Hopfield Networks

Aakash Kumar, Anatoly Khina, Frederik Mallmann-Trenn, Emanuele NatalearXiv 2026 · arXiv

TL;DR

This paper demonstrates that classical synchronous asymmetric Hopfield networks can store a superpolynomial number of long temporal sequences without needing complex nonlinearities or continuous states. By introducing a simple, robust "block-cyclic" network architecture, the authors achieve exponential-like sequence capacity and length while maintaining resilience to extreme noise. This bridges a long-standing gap between the structural simplicity of classical neural networks and the high-capacity demand of dynamic sequence processing.

What Problem Does This Paper Address?

The classical Hopfield network is a cornerstone of associative memory, but its traditional formulation relies on a symmetric weight matrix. While this symmetry guarantees stability, it restricts the network's behavior to static "point attractors"—meaning it can only store fixed patterns, not sequences.

To store temporal sequences, researchers turn to asymmetric Hopfield networks, which can transition between states in periodic loops called limit-cycle attractors. However, achieving a high storage capacity for long sequences within the classical binary-neuron framework has been an open challenge. Previous solutions either required highly engineered, fragile weight matrices or abandoned the classical model entirely in favor of "Modern Hopfield Networks" that use complex, non-biological nonlinear interactions.

The authors ask a fundamental question: Can a classical, structurally simple asymmetric Hopfield network robustly store a large number of long sequences?

Methods

The authors tackle this challenge by designing a novel Block-Cyclic Architecture (BCA). Instead of fine-tuning individual synaptic weights, they impose a coarse structural motif on the network topology:

  1. Dense Block-Cyclic Architecture: Neurons are grouped into disjoint "blocks" of size $d$. These blocks are connected in directed loops (block cycles) of varying lengths. Within a block cycle, every neuron in one block connects to every neuron in the next block with a weight of 1. Neurons within the same block have no internal connections.
  2. Sparse Block-Cyclic Architecture: To increase biological plausibility, the authors relax the all-to-all block connectivity. In this variant, each neuron only receives a small, odd number ($h \ge 3$) of random incoming edges from the preceding block.

To analyze this architecture, the authors combine tools from three distinct fields:

  • Combinatorics: Used to count the number of valid, non-repeating (aperiodic) binary patterns across the block cycles.
  • Number Theory: Used to determine the global period of the network, which equals the least common multiple (lcm) of the individual block cycle lengths.
  • Opinion Dynamics: The authors cleverly map the network's error-correction behavior to distributed consensus protocols (like the Gossip model). Neurons in a block act like "agents" voting on an opinion, allowing the network to leverage known robustness guarantees from distributed computing.

Datasets

Not discussed in the paper. As a theoretical machine learning paper, it does not use standard empirical datasets. Instead, the authors validate their mathematical proofs using synthetic finite-size simulations.

Key Insights and Ideas

  • Coarse Architecture over Fine-Tuning: High sequence capacity does not require precisely engineered weights or complex activation functions. Simple, macro-level structural motifs (directed loops of neuron blocks) are sufficient.
  • The Power of the LCM: By populating the network with block cycles of randomly sampled lengths, the global state's period scales with the least common multiple of those lengths. Number theory dictates that the lcm of a random set of integers grows exponentially, yielding superpolynomially long sequences.
  • Neurons as Voting Agents: Viewing the synchronous update rule as a majority-vote consensus mechanism allows the network to tolerate massive amounts of noise. If a block maintains even a slight majority bias toward its correct state, the entire block will self-correct in the next timestep.

Key Results

The paper provides the first mathematical proof that a classical synchronous asymmetric Hopfield network with $n$ neurons can achieve:

  • Superpolynomial Capacity: The network supports at least $\exp\left(\Omega\left(\frac{n}{(\log n)^2}\right)\right)$ distinct limit-cycle attractors.
  • Superpolynomial Period Length: Each stored sequence has a period of at least $\exp\left(\Omega\left(\frac{\sqrt{n}}{\log n}\right)\right)$.
  • Extreme Noise Robustness: The stored sequences are robust against random independent node flips with a probability up to $\frac{1}{2} - o(1)$ (approaching a completely random coin flip).
  • Structural Resilience: These properties hold true even when the network is highly sparsified or corrupted by adversarial connections designed to disrupt the cycle.
+--------------------------------------------------------------------------+
|                               MAIN RESULTS                               |
+-----------------------+--------------------------------------------------+
| Number of Limit Cycles| exp(Ω(n / (log n)²))                             |
+-----------------------+--------------------------------------------------+
| Period Length         | exp(Ω(√n / log n))                               |
+-----------------------+--------------------------------------------------+
| Noise Tolerance       | Flip probability up to 1/2 - o(1)                |
+-----------------------+--------------------------------------------------+

Limitations and Future Work

While the results are theoretically groundbreaking, the authors note a few key limitations:

  • Prescriptive Topology: The construction relies on a pre-defined, rigid block-cycle organization. The paper does not provide a local, biologically plausible learning rule (like Hebbian learning) that could naturally self-assemble this topology from an unstructured network.
  • Synchronous Updates Only: The proofs rely on parallel, synchronous updates (Little's rule). Whether these superpolynomial capacities persist under asynchronous, single-neuron updates remains an open question.
  • Lack of Real-World Benchmarking: The experimental validation is limited to synthetic simulations rather than task-level sequence benchmarks or biological neural recordings.

Summary updated

Abstract

Classical Hopfield networks are limited to static patterns due to symmetric weights, whereas asymmetric networks can encode temporal sequences via limit-cycle attractors. Achieving high-capacity storage of long sequences in classical synchronous asymmetric networks, however, has remained a challenge. We present a simple and robust construction within the classical asymmetric Hopfield model with binary neurons and synchronous updates, that allows $n$ neurons to support $\exp\!\big(Ω(n/(\log n)^2)\big)$ distinct limit-cycle attractors, each with period $\exp\!\big(Ω(\sqrt n/\log n)\big)$ and robust to random noise with flip probability up to $\frac12-o(1)$, yielding superpolynomial capacity in both the number and length of stored sequences. This is the first demonstration of such capacity for asymmetric Hopfield networks, which we obtain by combining results from combinatorics, number theory and the analysis of opinion dynamics. Our findings show that synchronous asymmetric Hopfield networks possess a sequence-memory capacity which is larger and more robust than previously recognized, demonstrating that, in both biological and artificial neural systems, robust sequence representation can be achieved through coarse architectural motifs rather than complex nonlinearities.
BibTeX
@misc{kumar2026beyond,
  author = {Aakash Kumar and Anatoly Khina and Frederik Mallmann-Trenn and Emanuele Natale},
  title = {{Beyond Fixed Points: Superpolynomial Capacity of Asymmetric Hopfield Networks}},
  year = {2026},
  eprint = {2605.24611},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2605.24611},
}

Loading related content...

Figure