Abstract: Heretofore, neural networks with external memory are restricted to single memory with lossy representations of memory interactions. A rich representation of relationships between memory pieces urges a high-order and segregated relational memory. In this paper, we propose to separate the storage of individual experiences (item memory) and their occurring relationships (relational memory). The idea is implemented through a novel Self-attentive Associative Memory (SAM) operator. Found upon outer product, SAM forms a set of associative memories that represent the hypothetical high-order relationships between arbitrary pairs of memory elements, through which a relational memory is constructed from an item memory. The two memories are wired into a single sequential model capable of both memorization and relational reasoning. We achieve competitive results with our proposed two-memory model in a diversity of machine learning tasks, from challenging synthetic problems to practical testbeds such as geometry, graph, reinforcement learning, and question answering.
Introduction
Presented at ICML 2020, this research introduces a novel memory architecture inspired by the functional neuroanatomy of the human brain. Specifically, the model draws on the division of labor between the perirhinal cortex, which handles individual items, and the hippocampus, which manages the complex relationships between them. The resulting SAM-based Two-Memory Model (STM) explicitly separates "item memory" from "relational memory" to overcome the limitations of traditional memory-augmented neural networks.
The Problem: Lossy Relational Representations
Traditional Memory-Augmented Neural Networks (MANNs), such as Neural Turing Machines (NTM) or Differentiable Neural Computers (DNC), typically rely on a single, flat memory structure. While effective for simple memorization, these models struggle with relational reasoning—the ability to understand how different pieces of information interact.
A primary bottleneck is the standard attention mechanism used in Transformers and other modern architectures. Dot-product attention reduces the relationship between two vectors to a single scalar value. The authors argue this representation is "coarse" and lossy. To solve complex problems—such as navigating a graph or solving a logic puzzle—a model requires a way to store and manipulate the structure of relationships separately from the content of the items themselves.
Methods: The STM Architecture
The core innovation of this work is the Self-attentive Associative Memory (SAM) operator and the dual-memory architecture it enables.
Outer Product Attention (OPA)
Unlike standard Dot Product Attention (DPA), OPA utilizes the outer product of a query-key pair and a value. This creates a matrix (or higher-order tensor) representation of the relationship, preserving bit-level interactions that a scalar dot product would discard. This mechanism is mathematically linked to Hebbian learning principles, where "cells that fire together, wire together."
Dual-Memory System
As shown in Figure 1, the STM architecture consists of two primary components:
- Item Memory ($M^i$): A gated auto-associative memory that stores individual experiences over time.
- Relational Memory ($M^r$): A 3D tensor constructed from the item memory via the SAM operator. It stores the "hypothetical high-order relationships" between arbitrary pairs of memory elements.
The model employs "distillation" and "transfer" operations to move knowledge between these two memories, allowing the relational structure to inform item retrieval and vice versa.
Experimental Evaluation
The STM model was evaluated across a diverse suite of benchmarks, demonstrating superior performance in tasks requiring both long-term memory and complex reasoning.
Algorithmic and Relational Tasks
The model was tested on standard algorithmic tasks like Copy and Priority Sort, as well as a new, more difficult task called Relational Associative Recall (RAR). In these tests, STM demonstrated significantly faster convergence and lower error rates compared to LSTMs, NTMs, and Relational Memory Cores (RMC).
Key Results
- bAbI Question Answering: STM achieved a new state-of-the-art mean error of 0.39% across the 20 tasks in the bAbI-10k suite, solving the entire set with remarkably low variance.
- Geometric & Graph Reasoning: In NP-hard problems like the Traveling Salesman Problem (TSP), STM significantly outperformed DNC and RMC, approaching optimal solutions without requiring specialized search algorithms like beam search.
- Reinforcement Learning Robustness: In Atari Pong, the model was tested under "frame skipping" conditions. While standard LSTMs failed as the gap between observations increased (up to 32 frames), STM maintained high performance, proving its dual-memory system is better at maintaining a coherent world model over long temporal gaps.
Discussion and Future Work
The separation of concerns between item and relational data allows for more specialized and efficient processing. By representing relationships as matrices rather than scalars, the model maintains a much richer representational capacity.
However, challenges remain regarding computational scaling. While OPA is powerful, the memory requirements for 3D tensors grow quickly. Future work may involve exploring low-rank approximations to scale to larger feature vectors. Additionally, the authors note that formally defining and automatically detecting the "order" of relationships required for specific tasks remains an open research question.
Conclusion
The SAM-based Two-Memory Model represents a significant step forward in memory-augmented neural networks. By mimicking the brain's segregation of item and relational information and utilizing high-order tensor representations, STM provides a robust framework for complex relational reasoning across a wide array of machine learning domains.
Summary updated