Indirect Encoding via Coordinate Based Generative Functions

47 concepts29 papers1 proposal

When optimizing complex systems—whether they are the weight matrices of deep neural networks, the voxel grids of soft robots, or the high-resolution textures of procedural game content—a fundamental challenge arises: the curse of dimensionality.

If we attempt to optimize every single parameter independently, a method known as direct encoding, the search space expands quadratically or exponentially with the size of the system. Furthermore, direct encoding treats every parameter as an isolated variable, completely ignoring the spatial and structural regularities—such as symmetry, repetition, and modularity—that characterize both natural organisms and effective engineered designs.

To overcome these limitations, a powerful paradigm has emerged: indirect encoding via coordinate-based generative functions. Instead of optimizing a massive array of explicit parameter values, we optimize a compact, secondary network that acts as a generative recipe. This recipe takes spatial coordinates as inputs and outputs the corresponding parameter values for those locations. By mapping geometry directly to functionality, this approach allows complex, large-scale structures to be compressed into a fraction of the parameters, enabling highly efficient and scalable optimization.

The Core Mechanism: Mapping Geometry to Parameters

To understand why coordinate-based generation works, consider the task of defining the connectivity of a large neural network. In a direct encoding scheme, a network with $N$ neurons requires optimizing an $N \times N$ weight matrix. If the network scales up, the number of parameters to optimize grows quadratically, quickly becoming intractable for evolutionary algorithms or global search methods (Igel et al.).

An indirect coordinate-based encoding shifts the problem entirely. As explored by Gauci et al., we can place the neurons within a geometric coordinate space (e.g., a 2D or 3D grid). To determine the weight of a connection between any two neurons, we pass their spatial coordinates into a continuous generator function, $f$:

$$\text{Weight} = f(x_1, y_1, x_2, y_2)$$

where $(x_1, y_1)$ is the position of the source neuron and $(x_2, y_2)$ is the position of the target neuron.

Direct Encoding:
[Parameter Matrix] ---> Massive, independent search space

Indirect Encoding:
(Spatial Coordinates) ---> [ Generative Function f ] ---> Parameter Value

Because the function $f$ is continuous, the underlying representation is entirely decoupled from the resolution of the final system. The same compact function can be queried to generate a $10 \times 10$ weight matrix or scaled up to generate a $1000 \times 1000$ matrix without adding a single parameter to the genotype. As Kunze et al. demonstrate, this strategy fundamentally alters the optimization landscape, reducing the dimensionality of the search space from quadratic to linear relative to the number of components.

Why It Works: Compositional Pattern-Producing Networks

The reason coordinate-based generation is uniquely effective lies in the mathematical structure of the generator function itself. The standard implementation of this concept is the Compositional Pattern-Producing Network (CPPN).

A CPPN is a directed acyclic graph of mathematical functions. Unlike traditional neural networks that rely almost exclusively on monotonic sigmoidal or rectified linear activations, CPPNs compose a diverse signature of activation functions, such as sines, cosines, Gaussians, and linear functions.

Each type of function introduces a specific geometric inductive bias:

  • Symmetric Functions (e.g., Gaussian, Absolute Value): If an early layer applies an absolute value function $|x|$ to an input coordinate, any downstream functions will receive identical values for $-x$ and $+x$. This naturally forces the final output to exhibit perfect reflectional symmetry.
  • Periodic Functions (e.g., Sine, Cosine): Passing a coordinate through a sine wave creates repeating patterns across space, allowing the system to automatically discover and exploit repetition and modularity.
  • Linear Functions: Linear mappings produce smooth gradients, which are essential for generating continuous transitions in material properties or weight strengths.

By composing these functions together, a CPPN can generate highly intricate, regular, and organic patterns from a minimal set of parameters. Liapis et al. show that when these networks are evolved against dynamic feature spaces, they naturally break initial representational biases to discover complex structural variations that direct pixel- or parameter-level manipulation would struggle to find.

Extreme Information Compression

Because geometric regularities allow for extreme information compression, coordinate-based generation functions act as powerful regularizers. They restrict the search space to patterns that "make sense" geometrically, filtering out unstructured noise.

A striking example of this compression is documented by Fernando et al., who introduced Differentiable Pattern Producing Networks (DPPNs) to optimize neural architectures via gradient descent. The authors successfully compressed a fully connected autoencoder containing 157,000 parameters into a coordinate generator of fewer than 200 parameters. Remarkably, during optimization, the generator naturally rediscovered classic structural motifs of deep learning—such as local receptive fields and weight sharing—without any explicit human architectural design. The network discovered convolution simply because localized, repeating patterns are the most mathematically efficient way for a coordinate function to represent a solution.

This geometric awareness is equally potent when combined with evolutionary algorithms. In frameworks like HyperNEAT, a CPPN is evolved to generate the weights of a primary task-solving network (Ha et al.; Verbancsics et al.). Because the CPPN is inherently aware of spatial relationships, the resulting neural networks excel at tasks where the input domain contains strong geometric regularities, such as board games like checkers (Gauci et al.) or visual screen layouts in Atari games (Hausknecht et al.).

Generalization Across Domains

The principle of mapping spatial coordinates to properties extends far beyond neural network weights. Any problem domain that can be mapped to a coordinate space can leverage indirect geometric encoding.

Soft Robotics and Morphological Computation

In voxel-based soft robotics, designing a physical body requires specifying the material properties (e.g., rigid, soft, active actuator, or passive) for thousands of individual cells in a grid. By using a CPPN to map 3D coordinates $(x, y, z)$ to material types, evolutionary algorithms can generate high-resolution, organic body plans from a compact representation (Corucci et al.).

This spatial coordination allows for the emergence of complex behaviors driven entirely by physical shape. For instance, Mertan et al. demonstrate that coordinate-mapped material distributions can achieve "closed-loop morphological computation," where the physical arrangement of stimuli-responsive materials coordinates sensing and actuation without a centralized neural "brain." When physical damage occurs, altering or evolving the underlying spatial distribution allows robots to recover functionality by autonomously changing their physical shape (Buchanan et al.).

(x, y, z) Coordinates ---> [ CPPN ] ---> Material Type (Rigid / Soft / Actuator)
                                    ---> Actuation Timing (Phase Offset)

Procedural Content Generation and Computer Vision

In game design, coordinate-based networks are utilized to generate visually compelling assets on the fly. By mapping spatial inputs to visual properties, systems can generate complex 2D spaceships or particle-system weapons that exhibit structural symmetry and repetition, aligning with human aesthetic preferences while avoiding user fatigue during interactive evolution (Togelius et al.; Hastings et al.).

Interestingly, this inherent bias toward geometric regularity also highlights fundamental differences between human and machine vision. Nguyen et al. demonstrated that deep neural networks can be easily "fooled" into classifying abstract, highly organized geometric patterns generated by CPPNs as real-world objects with near-absolute confidence. The abstract images, while unrecognizable to humans, trigger strong activations in discriminative models precisely because the coordinate-based generator produces clean, repeating structural motifs that mimic the low-level features deep models rely on.

Summary

Coordinate-based generative functions succeed because they treat structure not as a collection of isolated values, but as a continuous mathematical landscape. By optimizing the rules that generate patterns rather than the patterns themselves, this approach provides a scalable, compressed, and geometrically intuitive framework for both neural and physical design. Whether breeding plastic neural networks capable of lifelong learning (Soltoggio et al.) or growing soft-bodied machines, the strategy remains the same: leverage geometry to conquer dimensionality.

Key Foundational Work

Synergies between Evolutionary and Neural Computation (2005) — contributed 2 concept elements to this direction.
Acquiring evolvability through adaptive representations (2007) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_compositional_pattern_producing_networks_71d986dc,
  title = {compositional pattern-producing networks},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/compositional-pattern-producing-networks/rdr_71d986dc43544649bba633b499750e82},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_compositional_pattern_producing_networks_71d986dc_v1,
  title = {compositional pattern-producing networks},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/compositional-pattern-producing-networks/rdr_71d986dc43544649bba633b499750e82/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake