Why do neural network policies fail when their environment gets bigger? If you train a pathfinding agent or a game-playing model on a $10 \times 10$ grid, it will often fail catastrophically when deployed on a $50 \times 50$ grid. As Nantas Nardelli et al. observed in Value Propagation Networks, standard architectures tend to memorize specific global spatial configurations instead of learning the fundamental logic of information propagation.
To achieve zero-shot transfer across spatial scales, we must change how networks perceive space. The core solution relies on a powerful combination of locality and translation invariance: designing models that learn local interaction rules rather than global coordinate maps.
The Power of Fully Convolutional Architectures
Many spatial problems—whether playing board games, navigating mazes, or controlling physical systems—are governed by local rules that do not change based on absolute coordinates. A knight moves the same way in the corner of a chessboard as it does in the center; fluid particles obey the same partial differential equations regardless of the channel size.
By eliminating fully connected layers and adopting fully convolutional architectures, models can process variable-sized inputs without any architectural modification. For instance, Chao Gao et al. demonstrated in Adversarial Policy Gradient for Alternating Markov Games that fully convolutional networks allow agents trained on small Hex boards to generalize their learned strategies to much larger boards without additional training. Because the convolutional weights are shared across space, the learned tactical primitives are inherently scale-invariant.
This concept extends beyond simple grids to diverse board geometries. Dennis J. N. J. Soemers et al. leveraged convolutional policy heads in Towards a General Transfer Approach for Policy-Value Networks to transfer tactical knowledge across entirely different board sizes and connectivity rules, proving that local spatial relationships can be decoupled from global dimensions.
Grid-Wise Control and Spatial Abstraction
When dealing with multiple agents or complex scenes, traditional agent-centric representations scale poorly because the action space grows with the number of entities. A more scalable approach is to treat the environment as a spatial grid and map states directly to actions via dense pixel-to-pixel predictions.
In Grid-Wise Control for Multi-Agent Reinforcement Learning in Video Game AI, Lei Han et al. introduced an encoder-decoder framework (GridNet) that operates much like an image segmentation model. By applying shared convolutional filters across the grid, the model generalizes local tactical behaviors to any location, enabling zero-shot transfer to larger battles with higher agent densities. Similarly, Yimin Tang et al. developed RAILGUN: A Unified Convolutional Policy for Multi-Agent Path Finding Across Different Environments and Tasks, using a translation-invariant U-Net architecture to predict movements for all agents simultaneously, allowing zero-shot scaling from small synthetic training grids to massive maps.
[Small Input Grid] ---> [Shared Conv Filters] ---> [Local Tactics Learned]
|
v (Zero-Shot Tiling)
[Massive Input Grid] -> [Shared Conv Filters] ---> [Identical Tactics Applied Everywhere]
This spatial framing forces the network to learn reusable spatial primitives. As shown by Hanmo Chen et al. in Emergent collective intelligence from massive-agent cooperation and competition, centralized pixel-to-pixel policies allow complex cooperative behaviors learned on small maps to transfer seamlessly to larger environments.
To further prevent coordinate memorization, networks can decompose sensory inputs into discrete spatial segments or focus entirely on relative features. Byoung-Tak Zhang et al. utilized a Multi-focus Attention Network for Efficient Deep Reinforcement Learning to learn relational policies that generalize across positions. For computer vision tracking applications, Eric Crawford et al. demonstrated in Exploiting Spatial Invariance for Scalable Unsupervised Object Tracking that tiling identical convolutional "discovery units" allows unsupervised tracking models to generalize to larger image dimensions zero-shot.
Local Receptive Fields and Modular Tiling
Another way to exploit locality is to bound the agent's observation window or deploy identical local controllers across an expanded domain. If an agent only sees its immediate surroundings and receives high-level directional cues, the absolute size of the map becomes irrelevant.
In Path Planning of Cleaning Robot with Reinforcement Learning, Woohyeon Moon et al. combined local observation windows with global directional hints, removing dependencies on fixed-size map inputs. In a hierarchical setup, Debraj Ray demonstrated in Navigation in Adversarial Environments Guided by PRA* and a Local RL Planner that training a local planner on abstract features (like obstacle proximity and enemy risk scores) rather than raw coordinates enables zero-shot generalization to massive, high-fidelity maps.
This principle is highly effective for physical systems governed by partial differential equations (PDEs), where interactions are fundamentally local. Sebastian Peitz et al. showed in Distributed Control of Partial Differential Equations Using Convolutional Reinforcement Learning that a control policy optimized on a small spatial domain can be scaled to a much larger domain zero-shot by simply tiling additional identical local agents across the expanded space.
Similarly, Luca Guastoni et al. utilized a computationally restricted minimal domain in Discovering drag reduction strategies in wall-bounded turbulent flows using deep reinforcement learning to train translation-invariant policies that deploy directly to larger, physically realistic fluid channels. For discrete systems, Beomseok Kang et al. designed Attentive Recurrent Neural Cellular Automata in Learning locally interacting discrete dynamical systems: Towards data-efficient and scalable prediction, applying identical local transition rules to every cell to scale predictions to grids 64 times larger than the training size.
Embedding Structural and Geometric Priors
While translation invariance handles shifts in position, true spatial scaling often requires the network to understand how information propagates across distances or how strategies rotate and reflect.
Peter Karkus et al. addressed this in QMDP-Net: Deep Learning for Planning under Partial Observability by embedding a differentiable POMDP solver directly into convolutional layers. Because the transition and observation logic is structurally encoded, the model can scale to larger environments zero-shot simply by increasing the number of planning iterations at inference time. Rather than memorizing specific layouts, the agent learns abstract map-reading principles, a phenomenon also highlighted by Gino Brunner et al. in Teaching a Machine to Read Maps With Deep Reinforcement Learning.
Furthermore, spatial environments possess geometric symmetries. Linfeng Zhao et al. integrated these priors in Integrating Symmetry into Differentiable Planning by replacing standard convolutions with steerable convolutions. By baking equivariance to rotations and reflections directly into the network architecture, their symmetry-aware planners achieve near-perfect success rates when generalizing to maps significantly larger than those seen during training.
Summary
Zero-shot spatial scaling succeeds because it aligns the inductive biases of the neural network with the fundamental physics of the environment: locality and symmetry. By forcing models to learn how local entities interact or how information propagates locally rather than where things are globally, these architectures decouple policy execution from the absolute dimensions of the workspace.
Key Foundational Work
Adversarial Policy Gradient for Alternating Markov Games (2018) — contributed 1 concept elements to this direction.
QMDP-Net: Deep Learning for Planning under Partial Observability (2017) — contributed 1 concept elements to this direction.
Teaching a Machine to Read Maps With Deep Reinforcement Learning (2018) — contributed 1 concept elements to this direction.