Abstract
Neuroscientists postulate 3D representations in the brain in a variety of different coordinate frames (e.g. 'head-centred', 'hand-centred' and 'world-based'). Recent advances in reinforcement learning demonstrate a quite different approach that may provide a more promising model for biological representations underlying spatial perception and navigation. In this paper, we focus on reinforcement learning methods that reward an agent for arriving at a target image without any attempt to build up a 3D 'map'. We test the ability of this type of representation to support geometrically consistent spatial tasks such as interpolating between learned locations using decoding of feature vectors. We introduce a hand-crafted representation that has, by design, a high degree of geometric consistency and demonstrate that, in this case, information about the persistence of features as the camera translates (e.g. distant features persist) can improve performance on the geometric tasks. These examples avoid Cartesian (in this case, 2D) representations of space. Non-Cartesian, learned representations provide an important stimulus in neuroscience to the search for alternatives to a 'cognitive map'.
Introduction: Challenging the Cognitive Map
For decades, the dominant hypothesis in neuroscience has been that the brain constructs an internal 3D "cognitive map"—an allocentric, Cartesian representation of the world similar to the Simultaneous Localization and Mapping (SLAM) algorithms used in robotics. However, the rise of deep reinforcement learning (RL) has introduced a compelling alternative: agents can navigate complex, photorealistic environments using only 2D visual inputs, without ever constructing an explicit map.
This research investigates whether these "map-less" representations contain implicit geometric information sufficient to support spatial reasoning. By comparing a state-of-the-art RL model with a hand-crafted "Relative Visual Direction" (RVD) model, the authors explore how high-dimensional visual feature vectors can encode the structure of space through motion parallax and landmark stability.
Methods: Comparative Models of Spatial Representation
The study evaluates two distinct approaches to spatial representation:
1. The Learned Model (RL)
The researchers utilized a Siamese deep neural network architecture based on ResNet-50, trained via reinforcement learning (Zhu et al., 2017). This model processes a "current view" and a "goal view" to output navigation actions. To probe the internal representation, a "Knowledge Decoder" (a multi-layer perceptron) was employed to attempt to extract $(x, y)$ coordinates from the network's 512-dimensional feature vectors.
2. The Hand-Crafted Model (RVD)
The "Relative Visual Direction" model records the angles between all pairs of visible points in a scene. Crucially, this model incorporates motion parallax—the phenomenon where the relative positions of objects change at different rates based on their distance as the observer moves. By filtering for "low-parallax" features (distant objects that change position slowly), the model creates a stable scaffold for spatial representation.
The primary metric for evaluation was interpolation: given the feature vectors for two locations (A and B), could the model accurately estimate the representation for the midpoint?
Results: Geometric Consistency and the Power of Parallax
The comparison revealed significant differences in how geometry is encoded:
- Transparency of Representation: In the RVD model, the Euclidean distance between feature vectors correlates almost perfectly ($R=0.99$) with physical distance. In contrast, the RL model's correlation is significantly weaker and highly sensitive to the agent's orientation and current goal.
- Decoding and Interpolation: The RVD model's feature space preserves environmental regularity; a t-SNE projection of its features closely mirrors the physical grid of the environment. Furthermore, interpolation error in the RVD model increases predictably with distance. The RL model, however, showed noisy interpolation results, with errors often biased by specific visual textures rather than geometric relationships.
- The Role of Parallax: Prioritizing distant, stable landmarks (low-parallax features) proved essential for robustness. The authors suggest that instead of a coordinate system, the brain might use a hierarchy of hypotheses: distant objects provide a coarse "where" (e.g., "in the valley"), while nearby, high-parallax objects refine the estimate to a specific location.
Discussion: Toward Biologically Plausible Navigation
While the RL model demonstrates that navigation is possible without an explicit map, its internal representations are heavily "tangled" with task-specific goals. The RVD model suggests a more geometrically consistent path forward: by leveraging motion parallax, representations can become more robust and structured without resorting to a Cartesian coordinate system.
Future work should focus on integrating these principles—specifically the distinction between stable and transient visual features—into deep RL architectures. This shift moves the dialogue away from the binary "map vs. no-map" debate and toward a nuanced understanding of how high-dimensional sensory data implicitly encodes the geometry of our world.
Summary updated