ConwayNet: Interpretability, Activation Coherence, and Noise Resilience via a Game-of-Life-Inspired Lateral Inhibition in Artificial Neural Networks

Yair Cohen, Scott M. GilmorearXiv 2026 · arXiv

TL;DR

ConwayNet introduces a fixed, Game-of-Life-inspired "lateral inhibition" mechanism into neural network layers. By forcing local inputs to regulate outputs through differentiable "Birth" and "Survival" gates, the authors create networks with spatially coherent representations that are significantly more interpretable, reproducible, and resilient to noise.

What Problem Does This Paper Address?

Deep neural networks are powerful "universal function approximators," but they are often criticized as "black boxes." Their internal activations frequently look like high-frequency "salt-and-pepper" noise, making them difficult to interpret mechanistically. Furthermore, they are notoriously sensitive to pixel-level noise and can exhibit high "churn"—where two networks trained from nearly identical starting points disagree on a significant number of test images.

In contrast, Cellular Automata (CA) like Conway’s Game of Life (GoL) use simple, local rules to generate complex, programmable, and highly interpretable dynamics. This paper attempts to bridge these two worlds by using GoL-style rules to regulate the flow of information within standard neural network architectures.

Methods

The authors propose a mechanism called lateral inhibition. Unlike standard layers where every parameter is learned, this mechanism uses a fixed, non-learned rule based on neighborhood occupancy.

  1. The Standard Path: An input $x$ undergoes a standard affine transformation and activation (e.g., tanh) to produce a candidate activation $z$.
  2. The CA Path: In parallel, the input $x$ is used to calculate the "occupancy" $m$ of a unit’s local neighborhood (a $3 \times 3$ Moore neighborhood).
  3. Birth and Survival Gates: Using smooth, differentiable versions of GoL rules (specifically the B5/S3 rule), the model calculates "Birth" and "Survival" gates based on $m$.
  4. The Blend: The final output $y$ is a state-conditioned blend of these gates. If the unit’s internal state $z$ is "off," the Birth gate dominates; if it is "on," the Survival gate dominates.
Figure 1: Overview of the mechanism. The layer input x feeds both the standard path (affine transformation, normalisation, and bounded activation producing z) and the Game-of-Life-inspired lateral inhibition that regulates z into the output y via input-conditioned Birth and Survival gates. Colours mark the value range of each quantity: pink boxes are network states bounded in [−1, 1] (input, activation, output), blue boxes are CA states bounded in [0, 1] (neighbourhood mean, gates, and their normalised blend), and orange boxes are the transformations that map between the two ranges. The symbols are defined in Section 2.2 and Appendix A.

This mechanism is architecture-agnostic and was tested on Multi-Layer Perceptrons (MLPs), Convolutional Neural Networks (CNNs), Wide Residual Networks (WRNs), and Graph Convolutional Networks (GCNs).

Datasets

The authors evaluated ConwayNet across several standard computer vision benchmarks and their "color" variants:

  • MNIST-family: MNIST, EMNIST, FashionMNIST, and KMNIST.
  • CIFAR-10: Used for testing on more complex natural images with Wide Residual Networks.
  • Corrupted Data: The models were tested against Gaussian, shot, and impulse noise using a protocol similar to the ImageNet-C benchmark to evaluate out-of-distribution robustness.

Key Insights and Ideas

  • Fixed Rule, Learned Steering: Even though the GoL rule is fixed and receives no gradients, it shapes training. The network learns to "steer" its activations into regions where the fixed rule allows them to persist.
  • Spatial Coherence: In a standard MLP, hidden activations look like random noise. In ConwayNet, activations are smooth, contiguous, and clearly resemble the input object (e.g., the digit "2").
  • Scale Separation: The authors found that coarse structures persist through the depth of the network, while fine details are "renewed" at every layer. This mimics the separation of scales seen in physical systems like fluid dynamics.
  • The Co-activation Graph: By analyzing which units fire together, the authors found that ConwayNet naturally organizes itself into a spatial graph that recovers the geometry of the input image, unlike "plain" networks which have no such geometric structure.
Comparison of activation maps and classifier weights between ConwayNet and a plain network

Key Results

  • Noise Resilience: ConwayNet shows dramatic improvements in noise robustness. On EMNIST, the lateral inhibition CNN improved accuracy under noise by over 30 percentage points compared to a standard CNN.
  • Interpretability: Saliency maps (which show what the network is "looking at") are much cleaner. In ConwayNet, the gradient saliency rests almost entirely on the object itself rather than the background.
  • Reproducibility: Under label noise, ConwayNet reduced "churn" (disagreement between nearly identical models) by a factor of thirty.
  • Performance Trade-off: These benefits come at a small cost to "clean" accuracy (usually 1-2 points), though the Wide Residual Network actually saw a slight gain (+0.8 points) on CIFAR-10.
Percolation of the co-activation graph, showing how ConwayNet recovers image geometry

Limitations and Future Work

  • Training Difficulty: ConwayNets are harder to train; they converge more slowly and often require longer training schedules.
  • Sensitivity to Normalization: The mechanism is sensitive to how inputs are normalized, as the "occupancy" bands are fixed.
  • Manual "Wiring": Integrating the mechanism into new architectures currently requires manual effort to determine where to read inputs and how to reduce channels.
  • Rule Search: The authors primarily used the B5/S3 rule. There are over 262,000 possible "Life-like" rules; exploring this space with evolutionary algorithms or foundation models is a suggested path for future research.

Summary updated

Abstract

Deep neural networks achieve universal function approximation by stacking interconnected nonlinearities, but as a consequence they are sensitive to noise and resist mechanistic interpretation. In contrast, the local update rules of cellular automata, such as Conway's Game of Life, give rise to not merely interpretable but programmable dynamics. We explore a bridge between these worlds: a lateral inhibition in neural networks based on continuous Game-of-Life rules that derives inhibitory and excitatory responses from the layer input and blends them into the activation map in a state-conditioned way. The mechanism is differentiable and architecture-agnostic, and its regulation is imposed rather than learned: the rule's parameters are fixed and are not optimised, while the rule's values and derivatives shape training throughout the network. Wired into fully connected, convolutional, wide-residual and graph convolutional networks, it gives rise to several properties. Activation maps become coherent across space, depth and seeds, with coarse structure persisting through depth while fine detail is renewed at every layer. The co-activation graph of the hidden units departs from an Erdős--Rényi random graph: it is laid out on the image and its giant component grows in steps, as in a spatial graph. Networks started from nearly identical initialisations, under label noise, disagree on thirty times fewer test images. Saliency maps rest on the object rather than the background, and resilience to pixel noise not seen in training is substantially improved.
BibTeX
@misc{cohen2026conwaynet,
  author = {Yair Cohen and Scott M. Gilmore},
  title = {{ConwayNet: Interpretability, Activation Coherence, and Noise Resilience via a Game-of-Life-Inspired Lateral Inhibition in Artificial Neural Networks}},
  year = {2026},
  eprint = {2611.10760},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.10760},
}
Figure