Abstract
Navigation is crucial for animal behavior and is assumed to require an internal representation of the external environment, termed a cognitive map. The precise form of this representation is often considered to be a metric representation of space. An internal representation, however, is judged by its contribution to performance on a given task, and may thus vary between different types of navigation tasks. Here we train a recurrent neural network that controls an agent performing several navigation tasks in a simple environment. To focus on internal representations, we split learning into a task-agnostic pre-training stage that modifies internal connectivity and a task-specific Q learning stage that controls the network's output. We show that pre-training shapes the attractor landscape of the networks, leading to either a continuous attractor, discrete attractors or a disordered state. These structures induce bias onto the Q-Learning phase, leading to a performance pattern across the tasks corresponding to metric and topological regularities. By combining two types of networks in a modular structure, we could get better performance for both regularities. Our results show that, in recurrent networks, inductive bias takes the form of attractor landscapes -- which can be shaped by pre-training and analyzed using dynamical systems methods. Furthermore, we demonstrate that non-metric representations are useful for navigation tasks, and their combination with metric representation leads to flexibile multiple-task learning.
Introduction
In both neuroscience and AI, the "cognitive map" is often assumed to be a metric, Euclidean representation of space. However, different navigation tasks require different strategies. Finding a shortcut in a cluttered room requires precise metric coordinates, while following a sequence of landmarks to a destination is a topological task that doesn't strictly need a coordinate system. This study addresses how different internal representations, or inductive biases, affect an agent's ability to learn specific navigation tasks, bridging machine learning performance with dynamical systems theory.
Methods and Experimental Design
The researchers utilized a two-stage learning framework to isolate the effects of internal representations. First, in a task-agnostic pre-training stage, the internal connectivity of a recurrent neural network (RNN) was sculpted to represent either position (PosNet) or landmark memory (MemNet), while a control group (RandNet) retained random weights. Second, in a task-specific Q-learning stage, the internal weights were frozen, and only the readout weights were trained for specific navigation tasks in a 2D arena.
The tasks included basic reward reaching, navigating around obstacles (Hole/Bar), handling environment scaling (Scale), and integrating implicit context from wall colors. Fixed-point analysis was used to visualize the resulting neural dynamics and identify "slow points" where activity stabilizes.
Results: Attractor Landscapes and Performance Trade-offs
The study found that pre-training creates specific "shapes" or attractor landscapes in the RNN's phase space. PosNet developed continuous attractors, allowing for smooth coordinate representation and robustness to metric changes like new obstacles. In contrast, MemNet developed discrete attractors that acted as "bins" for landmark identities, making it superior at topological tasks where exact coordinates are less critical than sequences of events.
A fundamental trade-off was observed: metric-optimized networks (PosNet) struggled with environment scaling, while topological-optimized networks (MemNet) struggled with obstacles. However, a modular architecture that could switch between these biases outperformed any single-bias network across all tasks.
Discussion and Limitations
This work demonstrates that inductive bias in RNNs is physically encoded in their dynamical attractor landscapes. While the current study uses simplified 2D environments and manual pre-training, it provides a theoretical bridge between "black box" neural networks and the geometry of dynamical systems. Future work could explore how these structures emerge naturally in more complex, high-dimensional environments and how larger-scale models might exhibit hybrid attractor structures.
Summary updated