Implementing Inductive bias for different navigation tasks through diverse RNN attractors

Tie Xu, Omri Barak

Abstract

Navigation is crucial for animal behavior and is assumed to require an internal representation of the external environment, termed a cognitive map. The precise form of this representation is often considered to be a metric representation of space. An internal representation, however, is judged by its contribution to performance on a given task, and may thus vary between different types of navigation tasks. Here we train a recurrent neural network that controls an agent performing several navigation tasks in a simple environment. To focus on internal representations, we split learning into a task-agnostic pre-training stage that modifies internal connectivity and a task-specific Q learning stage that controls the network's output. We show that pre-training shapes the attractor landscape of the networks, leading to either a continuous attractor, discrete attractors or a disordered state. These structures induce bias onto the Q-Learning phase, leading to a performance pattern across the tasks corresponding to metric and topological regularities. By combining two types of networks in a modular structure, we could get better performance for both regularities. Our results show that, in recurrent networks, inductive bias takes the form of attractor landscapes -- which can be shaped by pre-training and analyzed using dynamical systems methods. Furthermore, we demonstrate that non-metric representations are useful for navigation tasks, and their combination with metric representation leads to flexibile multiple-task learning.

Introduction

In both neuroscience and AI, the "cognitive map" is often assumed to be a metric, Euclidean representation of space. However, different navigation tasks require different strategies. Finding a shortcut in a cluttered room requires precise metric coordinates, while following a sequence of landmarks to a destination is a topological task that doesn't strictly need a coordinate system. This study addresses how different internal representations, or inductive biases, affect an agent's ability to learn specific navigation tasks, bridging machine learning performance with dynamical systems theory.

Methods and Experimental Design

The researchers utilized a two-stage learning framework to isolate the effects of internal representations. First, in a task-agnostic pre-training stage, the internal connectivity of a recurrent neural network (RNN) was sculpted to represent either position (PosNet) or landmark memory (MemNet), while a control group (RandNet) retained random weights. Second, in a task-specific Q-learning stage, the internal weights were frozen, and only the readout weights were trained for specific navigation tasks in a 2D arena.

The two-stage learning framework (A) and task performance analysis (B-D).

The tasks included basic reward reaching, navigating around obstacles (Hole/Bar), handling environment scaling (Scale), and integrating implicit context from wall colors. Fixed-point analysis was used to visualize the resulting neural dynamics and identify "slow points" where activity stabilizes.

Results: Attractor Landscapes and Performance Trade-offs

The study found that pre-training creates specific "shapes" or attractor landscapes in the RNN's phase space. PosNet developed continuous attractors, allowing for smooth coordinate representation and robustness to metric changes like new obstacles. In contrast, MemNet developed discrete attractors that acted as "bins" for landmark identities, making it superior at topological tasks where exact coordinates are less critical than sequences of events.

A fundamental trade-off was observed: metric-optimized networks (PosNet) struggled with environment scaling, while topological-optimized networks (MemNet) struggled with obstacles. However, a modular architecture that could switch between these biases outperformed any single-bias network across all tasks.

Comparison of Metric vs. Topology scores shows the distinct performance profiles of PosNet and MemNet, and the superior flexibility of the Modular network.

Discussion and Limitations

This work demonstrates that inductive bias in RNNs is physically encoded in their dynamical attractor landscapes. While the current study uses simplified 2D environments and manual pre-training, it provides a theoretical bridge between "black box" neural networks and the geometry of dynamical systems. Future work could explore how these structures emerge naturally in more complex, high-dimensional environments and how larger-scale models might exhibit hybrid attractor structures.

Summary updated

Abstract

Navigation is crucial for animal behavior and is assumed to require an internal representation of the external environment, termed a cognitive map. The precise form of this representation is often considered to be a metric representation of space. An internal representation, however, is judged by its contribution to performance on a given task, and may thus vary between different types of navigation tasks. Here we train a recurrent neural network that controls an agent performing several navigation tasks in a simple environment. To focus on internal representations, we split learning into a task-agnostic pre-training stage that modifies internal connectivity and a task-specific Q learning stage that controls the network's output. We show that pre-training shapes the attractor landscape of the networks, leading to either a continuous attractor, discrete attractors or a disordered state. These structures induce bias onto the Q-Learning phase, leading to a performance pattern across the tasks corresponding to metric and topological regularities. By combining two types of networks in a modular structure, we could get better performance for both regularities. Our results show that, in recurrent networks, inductive bias takes the form of attractor landscapes -- which can be shaped by pre-training and analyzed using dynamical systems methods. Furthermore, we demonstrate that non-metric representations are useful for navigation tasks, and their combination with metric representation leads to flexibile multiple-task learning.
BibTeX
@inproceedings{xu2020implementing,
  author = {Tie Xu and Omri Barak},
  title = {{Implementing Inductive bias for different navigation tasks through diverse RNN attractors}},
  booktitle = {International Conference on Learning Representations},
  year = {2020},
  url = {https://openreview.net/forum?id=Y8D2A_-_Imn},
}

Loading related content...

Figure