Indirect Encoding of Neural Connectivity through Spatial Coordinates

This direction is merged into another direction: compositional pattern-producing networks. Its last version stays readable and citable here.

Version 1 of 114 papers19 conceptsfrozen 2026-10-04

In traditional machine learning, we often treat a neural network as a long, flat list of numbers. To improve the network, we tweak each individual weight. While effective for small models, this "direct encoding" approach struggles as systems grow. Nature doesn't work this way; the human brain has trillions of connections, yet our genome is far too small to specify each one. Instead, biology uses a generative process—a set of rules that "grow" a brain.

Research into indirect encoding seeks to replicate this efficiency. By using a secondary function to determine the weights of a primary network based on their spatial coordinates, researchers can evolve complex, large-scale architectures from a handful of parameters.

The Geometry of Intelligence

Most data is not a random collection of features; it has a shape. Images have local pixels and global symmetries; physical robots have left-right pairs. Jason Gauci and Kenneth O. Stanley argue in A Case Study on the Critical Role of Geometric Regularity in Machine Learning that ignoring these spatial relationships is a fundamental handicap.

When we treat a network as a flat vector, the algorithm has no way of knowing that two neurons are "next" to each other. By contrast, indirect encoding methods like HyperNEAT treat the network as a geometry. They use a generative function—often a Compositional Pattern Producing Network (CPPN)—that takes the $(x, y)$ coordinates of two neurons as input and outputs the weight of the connection between them.

Because the CPPN is a mathematical function, it can easily represent patterns like:

  • Symmetry: $f(x) = f(-x)$
  • Repetition: Periodic functions like $sin(x)$
  • Gradients: Smoothly increasing or decreasing values

This allows a very small "genotype" (the CPPN) to describe a massive "phenotype" (the neural network).

Scaling Through Generative Rules

The primary advantage of this approach is scalability. In Synergies between Evolutionary and Neural Computation, Christian Igel and Bernhard Sendhoff highlight that indirect encoding acts as a "global search" that discovers modular and hierarchical structures.

This scalability is most visible in memory-augmented systems. Jakob Merrild and Sebastian Risi demonstrated in HyperENTM: Evolving Scalable Neural Turing Machines through HyperNEAT that a controller evolved to manage a small memory bank can be instantly scaled to a massive one. Because the network "understands" the geometric pattern of memory access rather than specific memory addresses, it doesn't need to be re-trained when the memory size increases.

Beyond Neural Weights: Morphology and Growth

The principle of coordinate-based encoding extends beyond neural connections to the very "bodies" of artificial agents. In Evolutionary Developmental Soft Robotics, Francesco Corucci and Josh Bongard use functional genetic encoding to map spatial coordinates to material properties. This allows for the evolution of soft-bodied robots with complex, repetitive muscle structures using very few parameters.

Similarly, Yaochu Jin explores the idea of Evolutionary Developmental AI in Computational Evolution of Neural and Morphological Development, suggesting that we should optimize the rules of growth (regulatory rules) rather than the final components. This shifts the focus from designing a static machine to designing a developmental process that can self-organize into a sophisticated structure.

Hybridizing Evolution and Gradient Descent

While evolution is excellent at finding these high-level geometric patterns, it can be slow at fine-tuning specific values. Phillip Verbancsics and Josh Harguess propose a hybrid approach in Generative NeuroEvolution for Deep Learning. They use HyperNEAT to "grow" the initial connectivity and feature extractors of a deep network, exploiting spatial regularities in visual data, and then use traditional backpropagation to refine those weights.

This synergy suggests that the future of AI may not lie in choosing between evolution and gradient descent, but in using indirect encoding to define the topology and regularity of a system, while using local search to master the specific task at hand. By leveraging the geometry of the problem, we move from building "flat" models to building structured, scalable, and brain-like architectures.

Papers in this version

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_indirect_encoding_of_neural_connectivity_effc6da3,
  title = {Indirect Encoding of Neural Connectivity through Spatial Coordinates},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/indirect-encoding-of-neural-connectivity-through-spatial-coordinates/rdr_effc6da39d3f4960b956bc21510162e0},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_indirect_encoding_of_neural_connectivity_effc6da3_v1,
  title = {Indirect Encoding of Neural Connectivity through Spatial Coordinates},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/indirect-encoding-of-neural-connectivity-through-spatial-coordinates/rdr_effc6da39d3f4960b956bc21510162e0/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}