Prediction and Generalisation over Directed Actions by Grid Cells

Changmin Yu, Timothy E.j. Behrens, Neil Burgess

Abstract

Knowing how the effects of directed actions generalise to new situations is key to rapid generalisation across new situations. Markovian tasks can be characterised by a state space and a transition matrix. Recent work has proposed that neural grid codes provide an efficient representation of the state space as eigenvectors of a transition matrix reflecting diffusion across states, allowing for efficient prediction of future state distributions. Here, we extend the eigenbasis prediction model using tools from Fourier analysis to prediction over arbitrary translation-invariant directed transition structures. We show that a single set of eigenvectors can support predictions over arbitrary directed actions via action-specific eigenvalues. This framework unifies the role of the grid system in predictive planning, sense of direction, and path integration, supporting generalisable inference over directed actions across different tasks.

Introduction

Neuroscientists have long debated the primary function of entorhinal grid cells. Two dominant theories suggest they either perform path integration—calculating position by integrating velocity—or serve as predictive maps, providing a coordinate system for future state prediction. While eigenvector-based models have supported the predictive view, they have historically struggled to account for directed actions (e.g., moving North) without recomputing the entire map. We bridge these theories by demonstrating that grid cells act as a universal Fourier eigenbasis, where directed movements are represented as simple phase shifts.

Theoretical Framework: Grid Cells as a Fourier Basis

We treat a 2D environment as a circulant transition matrix. A fundamental property of circulant matrices is that their eigenvectors are always Fourier modes (plane waves). By maintaining a fixed set of eigenvectors (the grid cells) and simply updating their action-specific eigenvalues, the brain can predict the outcomes of various directed actions without changing the underlying neural representation.

Transition matrices (A, C, E) and their corresponding predictive distributions or Successor Representations (B, D, F). This illustrates how different action structures—from symmetric diffusion to directed movement—are captured within the same framework.

This approach utilizes the Fourier Shift Theorem, which states that a spatial displacement is equivalent to a phase shift in the frequency domain. This allows for path integration to be performed by re-weighting existing grid cell activity. Furthermore, we define a "sense of direction" as the optimal combination of actions to reach a target state, which can be derived analytically from this Fourier representation.

Results

Reinforcement Learning and Generalisation

To test the utility of this representation, we implemented a GridCell-DQN (gc-DQN) that incorporates Fourier-based state representations. In the CartPole-v1 control benchmark, the gc-DQN significantly outperformed standard DQNs, reaching higher reward plateaus much earlier in training. This suggests that the Fourier basis provides a powerful inductive bias for learning policies in continuous spaces.

Grid cell patterns (B, C) and their constituent Fourier plane waves (G). Panel F shows the relationship between the model's intuitive distance and true Euclidean distance, demonstrating the metric properties of the representation.

Biological Match

Our model successfully replicates biological phenomena such as theta phase precession—where neurons fire at progressively earlier phases of the 8Hz theta rhythm as an animal moves. This suggests that the brain may indeed be implementing the Fourier-like phase shifts described in our framework to update its internal spatial map.

Discussion

While our core mathematical framework relies on translation invariance, we demonstrate that the model can handle environmental complexities like obstacles using the Woodbury matrix inversion formula. This allows for rapid local updates to the map without requiring global retraining. Future research will investigate how this Fourier approach scales to high-dimensional, non-spatial abstract reasoning tasks and the specific synaptic mechanisms that might implement action-dependent eigenvalue updates in biological circuits. By unifying predictive planning and path integration, we offer a comprehensive account of the grid system's role in cognitive mapping.

Summary updated

Abstract

Knowing how the effects of directed actions generalise to new situations (e.g. moving North, South, East and West, or turning left, right, etc.) is key to rapid generalisation across new situations. Markovian tasks can be characterised by a state space and a transition matrix and recent work has proposed that neural grid codes provide an efficient representation of the state space, as eigenvectors of a transition matrix reflecting diffusion across states, that allows efficient prediction of future state distributions. Here we extend the eigenbasis prediction model, utilising tools from Fourier analysis, to prediction over arbitrary translation-invariant directed transition structures (i.e. displacement and diffusion), showing that a single set of eigenvectors can support predictions over arbitrary directed actions via action-specific eigenvalues. We show how to define a "sense of direction" to combine actions to reach a target state (ignoring task-specific deviations from translation-invariance), and demonstrate that adding the Fourier representations to a deep Q network aids policy learning in continuous control tasks. We show the equivalence between the generalised prediction framework and traditional models of grid cell firing driven by self-motion to perform path integration, either using oscillatory interference (via Fourier components as velocity-controlled oscillators) or continuous attractor networks (via analysis of the update dynamics). We thus provide a unifying framework for the role of the grid system in predictive planning, sense of direction and path integration: supporting generalisable inference over directed actions across different tasks.
BibTeX
@misc{yu2020prediction,
  author = {Changmin Yu and Timothy E.J. Behrens and Neil Burgess},
  title = {{Prediction and Generalisation over Directed Actions by Grid Cells}},
  year = {2020},
  eprint = {2006.03355},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2006.03355},
}
Figure