Abstract
This paper presents a novel associative memory model utilizing relaxation labeling processes, effectively overcoming the symmetry constraints of traditional Hopfield networks. By framing memory retrieval as a consistency-seeking process rather than energy minimization, the model supports asymmetric connections and multivalued patterns. Experimental results on binary digit recognition demonstrate superior error-correction capabilities and robustness to noise compared to standard Hebbian models.
Introduction
Since the early 1980s, the Hopfield network has served as the foundational model for associative memory—the ability to retrieve a full pattern from a partial or noisy input. However, it possesses a significant biological and practical limitation: the requirement for perfectly symmetric connections. In a Hopfield network, the strength of the connection from neuron A to neuron B must be identical to the connection from B to A, a condition rarely met in biological neural systems.
This paper introduces an alternative framework based on Relaxation Labeling Processes (RLP), a staple of computer vision. By leveraging RLP, the authors develop a system that naturally supports asymmetric weights and multivalued patterns, achieving superior performance in error correction.
Methodology
The authors propose using RLP as the dynamical engine for associative memory. In this framework, the model consists of "objects" (positions in a pattern) that are assigned "labels" (the value at that position).
Unlike traditional models that minimize an energy function, RLP focuses on consistency. A pattern is considered consistent if the label assigned to each object receives the highest possible support from its neighbors. The core innovation is the use of a compatibility matrix ($R$) that does not have to be symmetric. The authors demonstrate that even without symmetry, "strictly consistent" labelings act as local attractors, ensuring the system converges toward stored memories during relaxation.
To "load" patterns into the memory, the authors treat learning as an optimization problem. They use Eremin’s algorithm, an iterative method for solving systems of linear inequalities, to find the optimal compatibility matrix that ensures desired patterns become stable attractors.
Experimental Setup
The model was evaluated using a classic benchmark: a set of ten binary digits represented on a $5 \times 5$ grid.
To test error correction, the authors generated perturbed versions of these digits by flipping bits (measured by Hamming distance) and adding Gaussian noise. The results were compared against standard Hebb-trained Hopfield networks and optimal perceptron-based learning procedures.
Results and Discussion
The simulation results demonstrated that the relaxation labeling model is highly effective for error correction. The model significantly outperformed standard Hebb-trained networks and remained competitive with more complex perceptron-based rules.
As shown in the figure, the network maintains a high percentage of successful recalls even at a Hamming distance of 3 or 4 (representing 12-16% of bits flipped). This robustness suggests that dynamical stability in associative memory does not strictly require symmetric weights.
Conclusion
By decoupling associative memory from the necessity of symmetric weights, this model offers a more flexible and biologically inspired approach. While future work is needed to rigorously estimate storage capacity and evaluate the frequency of spurious states, the RLP-based model represents a significant advancement in robust, multivalued associative memory systems.
Reference: Pelillo, M., & Fanelli, A. M. (1995). An asymmetric associative memory model based on relaxation labeling processes. ESANN'1995 proceedings.
Summary updated