Wave Propagation Phenomena in Nonlinear Hierarchical Neural Networks with Predictive Coding Feedback Dynamics

Andrea Alamia, Léa Dalliès, Grégory Faye, Rufin VanRullen

Abstract

We propose a mathematical framework to systematically explore the propagation properties of a class of continuous-in-time nonlinear neural network models comprising a hierarchy of processing areas, mutually connected according to the principles of predictive coding. We precisely determine the conditions under which upward propagation, downward propagation, or even propagation failure can occur in both bi-infinite and semi-infinite idealizations of the model. We also study the long-time behavior of the system when either a fixed external input is constantly presented at the first layer of the network or when this external input consists of the presentation of constant input with large amplitude for a fixed time window followed by a reset to a down state of the network for all later times. In both cases, we numerically demonstrate the existence of threshold behavior for the amplitude of the external input characterizing whether or not a full propagation within the network can occur. Our theoretical results are consistent with predictive coding theories and allow us to identify regions of parameters that could be associated with dysfunctional perceptions.

Introduction

Information processing in the brain is fundamentally hierarchical and bidirectional. According to Predictive Coding (PC) theory, higher cortical areas generate top-down "predictions" about sensory input, while lower areas transmit "prediction errors"—the difference between predictions and actual sensory data—back up the hierarchy. While PC has become a cornerstone of modern neuroscience and machine learning, the specific mathematical dynamics governing how these signals propagate through deep, nonlinear networks have remained largely unexplored.

Previous research focused on linear versions of these models, but biological systems are inherently nonlinear. This paper addresses this gap by introducing a continuous-time, nonlinear mathematical model to systematically analyze the conditions under which neural activity successfully travels through a hierarchical network.

Mathematical Framework and Methodology

The authors derive a continuous-in-time functional differential equation from discrete predictive coding update rules. Key components of the model include:

  • Nonlinearity and Bistability: By incorporating a sigmoidal activation function, the model allows network nodes to exist in two stable states: a "down" (resting) state and an "up" (highly active) state.
  • Structural Simplification: To ensure mathematical tractability, the study utilizes a "one-population model." In this setup, each neuron connects exclusively to its direct counterpart in the layers immediately above and below, using unit weights.
  • Traveling Wave Theory: The network is treated as an infinite or semi-infinite lattice. The researchers apply traveling wave theory to calculate the speeds at which the "up" state overtakes the "down" state (and vice versa).
  • Stimulus Modeling: External sensory inputs are simulated at the lowest layer as either constant signals or brief "flashes" to observe the resulting network dynamics over time.

Dynamics of Wave Propagation

The study reveals that information in predictive coding networks propagates as a moving front, or wave, switching sequential layers from resting to active states.

Propagation Failure and Unconscious Perception

A critical discovery is the existence of "pinning"—specific parameter regimes where the wave speed drops to zero. In these cases, the signal fails to propagate through the hierarchy. Biologically, this phenomenon maps to unconscious perception, where a stimulus is registered by early sensory areas but fails to reach higher cognitive centers.

Thresholds for Perception

The model demonstrates that triggering a full network cascade requires a stimulus to cross sharp, mathematically definable thresholds in both amplitude (intensity) and duration (temporal persistence).

Clinical Implications: The Autism-Schizophrenia Continuum

The balance between bottom-up (feedforward) and top-down (feedback) propagation speeds provides an intriguing framework for understanding psychiatric conditions.

  • Autism Spectrum: Regimes dominated by bottom-up propagation align with sensory-driven behaviors and hypersensitivity often seen in the autism spectrum.
  • Schizophrenia: Regimes dominated by top-down propagation mirror internally generated hallucinations, where predictions override sensory evidence, characteristic of schizophrenia.

Key Results

The authors successfully computed wave speeds ($c_{u \to d}$ and $c_{d \to u}$) as functions of feedforward error correction strength ($p$) and feedback error correction strength ($q$). When a stimulus is briefly flashed, the network exhibits three distinct behaviors:

  1. Stagnation: Activity uniformly decays back to the "down" state.
  2. Stacked Interface: Activity travels upward as a finite pulse (a wave of "up" state sandwiched between "down" states).
  3. Front Propagation: The "up" state permanently transitions the network layer by layer.

The research highlights that a "normal" working regime—where the network can successfully propagate both sensory signals and predictions—exists only within a narrow band of parameters.

Limitations and Future Directions

While the model provides significant insights, it relies on several simplifying assumptions:

  • Architectural Simplicity: The 1-to-1 connectivity between layers does not yet reflect the complex, dense connectivity of biological brains or standard deep learning architectures.
  • Lack of Adaptation: The current bistable system remains in the "up" state indefinitely once activated. Real neurons experience fatigue and adaptation.

Future work aims to extend this framework to multi-population models and incorporate "spike frequency adaptation." This would allow high-activity states to decay naturally, transforming permanent shifts into biologically plausible transient pulses.

Summary updated

Abstract

We propose a mathematical framework to systematically explore the propagation properties of a class of continuous in time nonlinear neural network models comprising a hierarchy of processing areas, mutually connected according to the principles of predictive coding. We precisely determine the conditions under which upward propagation, downward propagation or even propagation failure can occur in both bi-infinite and semi-infinite idealizations of the model. We also study the long-time behavior of the system when either a fixed external input is constantly presented at the first layer of the network or when this external input consists in the presentation of constant input with large amplitude for a fixed time window followed by a reset to a down state of the network for all later times. In both cases, we numerically demonstrate the existence of threshold behavior for the amplitude of the external input characterizing whether or not a full propagation within the network can occur. Our theoretical results are consistent with predictive coding theories and allow us to identify regions of parameters that could be associated with dysfunctional perceptions.
BibTeX
@article{alamia2025wave,
  author = {Andrea Alamia and Léa Dalliès and Grégory Faye and Rufin VanRullen},
  title = {{Wave Propagation Phenomena in Nonlinear Hierarchical Neural Networks with Predictive Coding Feedback Dynamics}},
  journal = {Bulletin},
  year = {2025},
  doi = {10.1007/s11538-025-01492-4},
}
Figure