An Attractor Neural Network Model of Recall and Recognition

Eytan Ruppin, Yehezkel YeshurunNeurIPS 1990

Introduction

This paper demonstrates that a standard Hopfield-based Attractor Neural Network (ANN) can naturally replicate complex human memory phenomena. These include common psychological quirks, such as why we struggle to recall names as we age or why frequent words are easier to recall but harder to recognize. By defining "Recall" as successful convergence to a memory and "Recognition" as the speed of that convergence, the authors show that these behaviors are emergent properties of the network's architecture rather than the result of manually tuned parameters.

Bridging Psychology and Connectionism

In the early 1990s, a significant gap existed between "classical" psychological models of memory and the burgeoning field of connectionism (neural networks). Classical models often relied on explicitly assigned parameter values to match experimental data.

The authors tackle the challenge of showing that a single computational framework can qualitatively account for a wide range of human memory behaviors. Specifically, they investigate:

  • Recall: Retrieving an item given a cue (cued recall) or spontaneously (free recall).
  • Recognition: Identifying if an item was previously seen.

The underlying goal was to prove that these psychological observations are inherent to how distributed, associative neural systems function.

Methodology: The Hopfield Model

The researchers utilize a Hopfield Model, a type of recurrent neural network where memories are stored as "stable states" (attractors) within the network's energy landscape. The network uses $n=500$ neurons and stores simulated memory patterns as randomly generated binary vectors.

Key technical components include:

  • Dynamics: The network features non-linear, iterative, and asynchronous state updates, incorporating a "temperature" parameter ($T$) to represent stochastic noise.
  • Recall: Defined as successful if the network, starting from a noisy cue, converges to the correct stored memory pattern.
  • Recognition: Defined as successful if the network reaches any stable state within a specific time window. Learned memories have much deeper and wider "basins of attraction" than random noise, meaning the network "falls" into them much faster.
  • Context and Associations: "Context" is represented as a specific substring of the memory encoding. "Associational linkage" (how words relate to each other) is modeled as an external field vector ($E$) that influences the neurons.

Key Insights into Human Memory

Recognition is About Speed

One of the most elegant ideas presented is that recognition isn't just "finding" a memory, but rather how quickly the network stabilizes. If the input is familiar, the network snaps into a stable state almost instantly. If it's unfamiliar, the network wanders through "shallow" energy landscapes for a significantly longer time.

The Aging Brain as a Diluted Network

The authors model aging as "synaptic dilution"—the weakening or deletion of connections. They found that as connections are lost, the network first loses the ability to recall "far-away" or weakly cued patterns. However, the ability to recognize high-overlap patterns remains relatively intact. This perfectly mirrors how older adults often struggle with recall but perform well on recognition tasks.

The Word-Frequency Paradox

High-frequency words (like "house") are easier to recall but harder to recognize than low-frequency words (like "abacus"). The authors model this by assuming high-frequency words have more "associational links" (external fields). In recall, these links help pull the network toward the right memory. In recognition, these same links can create "post-optimal" interference, slowing down convergence.

Experimental Results

The List-Length Effect

As the number of stored memories (memory load) increases, the "basins of attraction" shrink. The simulations showed that as load increases, the number of iterations required for convergence grows exponentially, eventually leading to recognition failure.

Iterations required for convergence grow exponentially as memory load increases.

Recall Probability and Context Shift

The network's ability to successfully recall a memory is highly dependent on both the memory load and the initial overlap of the cue.

Recall probability as a function of memory load and initial overlap.

Furthermore, by flipping bits in the "context" portion of a cue, the authors showed that recall performance deteriorates significantly, even while the stability of the memories themselves remains intact. This explains why humans often forget things when moving to a different room or context.

Quantitative Alignment

The authors derived a formula for the expected number of iterations for convergence: $E(X) \approx n \cdot \ln(d)$. In simulations with $n=500$ neurons, the network converged in 300–400 iterations, precisely matching their mathematical predictions.

Limitations and Future Work

While the model is highly explanatory, it has limitations:

  • Non-Memory States: The network sometimes converges to "spurious" or non-memory states. While suggested to represent "recall failure," the exact psychological meaning of these states remains out of scope.
  • Simplistic Encodings: The model uses random binary patterns, whereas real human memory involves complex hierarchical and semantic structures.
  • Scaling: While the qualitative results hold for $n=500$, further quantitative examination is needed to see how these emergent properties scale to the billions of neurons in a human brain.

Summary updated

BibTeX
@inproceedings{ruppin1990attractor,
  author = {Eytan Ruppin and Yehezkel Yeshurun},
  title = {{An Attractor Neural Network Model of Recall and Recognition}},
  booktitle = {Advances in Neural Information Processing Systems},
  year = {1990},
}
Figure