Introduction
Understanding and predicting the trajectories of complex interacting systems—ranging from physical particles to human social dynamics—remains a fundamental challenge in machine learning. While models like Neural Relational Inference (NRI) and its dynamic counterpart (DNRI) have made strides in inferring latent interactions, they often struggle with long-range dependencies. This paper introduces MemDNRI, a framework that leverages dual external memory pools to overcome the "wash-out" effect inherent in standard recurrent architectures.
The Challenge of Latent Dynamics
In many real-world scenarios, the underlying relationships between entities are unobserved and change over time. For instance, in a basketball game, player movements are influenced by shifting tactical alignments. Previous state-of-the-art models like DNRI utilize LSTMs to track these changing relations. However, as LSTMs process new information, their internal hidden states tend to lose older, critical context. This limitation hinders the ability to model erratic movements or maintain accuracy over extended prediction horizons.
Proposed Method: MemDNRI
MemDNRI addresses the memory bottleneck by augmenting a Variational Autoencoder (VAE) architecture with two distinct associative memory pools:
- RelMem (Relation Memory): Stores information regarding the interactions (edges) between entities. To ensure scalability, this memory is shared across all edges, utilizing a "hitting rate" normalization technique to prevent data overwrites.
- EntMem (Entity Memory): Tracks the state and history of individual entities. Each entity is assigned a dedicated memory slot to avoid addressing conflicts.
The model employs asymmetrical addressing, using content-based attention for reading and location-based addressing for writing. These memory modules are integrated into Graph Neural Networks (GNNs), where the GNN handles message passing while the memory provides the necessary long-term context.
Experimental Evaluation
The authors evaluated MemDNRI across three diverse domains: synthetic physics simulations, CMU Motion Capture (MoCap) data, and real-world basketball trajectories.
MemDNRI demonstrated superior performance, achieving a relation estimation accuracy of 0.98 on synthetic data (compared to 0.97 for DNRI). Crucially, in the CMU jumping task—characterized by sudden, non-linear movements—MemDNRI maintained a stable Mean Squared Error (MSE) even as the prediction window expanded, whereas previous models saw their error rates explode as they "forgot" the initial state.
Key Insights: Global Latent Structures
A significant finding of this work is that the external memory acts as a global repository rather than a simple buffer. By visualizing the correlation between read and write operations, the authors observed that the model frequently "recalls" information from the very beginning of a sequence to inform predictions at the end.
This behavior confirms that MemDNRI captures long-term global structures. Furthermore, the separation of Relation and Entity memory allows the model to independently learn interaction patterns versus individual entity states, leading to more robust generalization.
Conclusion and Future Work
MemDNRI represents a significant step forward in modeling dynamic interacting systems by providing a mechanism for long-term dependency tracking. While the current implementation introduces some computational overhead and assumes a fully connected graph, future iterations could explore memory integration into the encoder and application to sparser, hierarchical structures.
Summary updated