Prediction with directed transitions: complex eigenstructure, grid cells and phase coding

Changmin Yu, Timothy Edward John Behrens, Neil Burgess

Abstract

Knowing how the effects of directed actions generalize to new situations (e.g., moving North, South, East, and West) is key to rapid generalization across new situations. Recent work has proposed that neural grid codes provide an efficient representation of state space as eigenvectors of a transition matrix reflecting diffusion. Here, we extend the eigenbasis prediction model to arbitrary translation-invariant directed transition structures (i.e., displacement and diffusion). We show that a single set of eigenvectors can support predictions over arbitrary directed actions via action-specific eigenvalues. This framework provides a unifying account of grid cell firing, "sense of direction," and path integration, while demonstrating that adding Fourier representations to deep reinforcement learning agents aids policy learning in continuous control tasks.

Introduction

Grid cells in the mammalian entorhinal cortex are widely regarded as a fundamental component of the brain's navigation system. Recent theoretical work has suggested that these cells represent the eigenvectors of a transition matrix, providing a basis set for "intuitive planning" or predicting future state distributions. However, a significant limitation of existing models is their reliance on undirected transitions, such as random diffusion. In real-world scenarios, actions are inherently directed (e.g., "move two steps East"). Standard eigendecomposition would require a unique set of eigenvectors for every possible action, a requirement that is both computationally prohibitive and biologically inconsistent with the observed stability of grid cell patterns across different environments.

The Fourier Eigenbasis for Directed Prediction

To address this, the authors leverage the property of translation invariance. In many environments, the effect of an action is independent of the starting location. Such translation-invariant transitions can be represented by circulant matrices. A key mathematical property of all circulant matrices is that they share a common set of eigenvectors: the Fourier modes (plane waves).

By adopting a Fourier-based approach, the model can maintain a fixed set of eigenvectors—the grid cells—while accounting for different directed actions through action-specific eigenvalues. This is achieved via the Fourier shift theorem, where velocity-driven updates are handled by modifying the phase of the eigenvalues rather than recomputing the basis. This allows a single neural representation to generalize across a wide variety of directed movements.

Figure 1: Visualization of transition matrices (A, C, E) and their corresponding predictive state distributions (B, D, F). The model demonstrates how different transition structures, including directed movement and diffusion, are captured within the framework.

Unifying Mechanistic Models

The proposed framework offers a mathematical bridge between disparate models of grid cell function. It proves the equivalence between the predictive eigenbasis model and traditional mechanistic theories:

  • Continuous Attractor Networks (CAN): The update dynamics of the predictive state distribution correspond to the movement of a "bump" of activity in a CAN.
  • Oscillatory Interference (OI): The Fourier components act as velocity-controlled oscillators, where interference patterns generate the characteristic grid firing.

Furthermore, the model naturally explains theta phase precession. As an animal moves through a place field, the predictive update of the state distribution causes the peak of activity to shift, resulting in firing at progressively earlier phases of the brain's theta rhythm. In this framework, precession is a direct byproduct of the predictive update process.

Results and Reinforcement Learning

The authors integrated this Fourier representation into a deep reinforcement learning agent, the gc-DQN. In the CartPole-v1 control task, the gc-DQN significantly outperformed standard DQNs and "Dyna-Q" agents, reaching higher rewards with fewer training episodes. This suggests that the grid-like Fourier basis provides a powerful inductive bias for learning value functions in continuous spaces.

The model also enables a mathematical "sense of direction," allowing agents to navigate efficiently in complex environments, such as "windy" grid worlds with external forces. By calculating the combination of actions that maximizes the probability of reaching a target, the agent can compensate for environmental deviations from perfect translation invariance. Additionally, local obstacles can be efficiently integrated into the global Fourier map using the Woodbury matrix identity, avoiding the need for a full recomputation of the basis.

Figure 6: (B) A 3D representation of the predictive landscape used for navigation. (C) The relationship between the model's "intuitive distance" and true Euclidean distance, showing how the representation captures spatial topology.

Conclusion

This work provides a unifying framework for the role of the grid system in predictive planning, sense of direction, and path integration. By framing grid cells as a universal Fourier eigenbasis, the authors demonstrate how the brain can support generalizable inference over directed actions across diverse tasks and environments, offering a compelling explanation for the stability and utility of grid codes in the mammalian brain.

Summary updated

Abstract

Knowing how the effects of directed actions generalise to new situations (e.g. moving North, South, East and West, or turning left, right, etc.) is key to rapid generalisation across new situations. Markovian tasks can be characterised by a state space and a transition matrix and recent work has proposed that neural grid codes provide an efficient representation of the state space, as eigenvectors of a transition matrix reflecting diffusion across states, that allows efficient prediction of future state distributions. Here we extend the eigenbasis prediction model, utilising tools from Fourier analysis, to prediction over arbitrary translation-invariant directed transition structures (i.e. displacement and diffusion), showing that a single set of eigenvectors can support predictions over arbitrary directed actions via action-specific eigenvalues. We show how to define a "sense of direction" to combine actions to reach a target state (ignoring task-specific deviations from translation-invariance), and demonstrate that adding the Fourier representations to a deep Q network aids policy learning in continuous control tasks. We show the equivalence between the generalised prediction framework and traditional models of grid cell firing driven by self-motion to perform path integration, either using oscillatory interference (via Fourier components as velocity-controlled oscillators) or continuous attractor networks (via analysis of the update dynamics). We thus provide a unifying framework for the role of the grid system in predictive planning, sense of direction and path integration: supporting generalisable inference over directed actions across different tasks.
BibTeX
@misc{yu2020prediction,
  author = {Changmin Yu and Timothy Edward John Behrens and Neil Burgess},
  title = {{Prediction with directed transitions: complex eigenstructure, grid cells and phase coding}},
  year = {2020},
  eprint = {2006.03355},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2006.03355},
}
Figure