Abstract
We propose a mathematical framework to systematically explore the propagation properties of a class of continuous in time nonlinear neural network models comprising a hierarchy of processing areas, mutually connected according to the principles of predictive coding. We precisely determine the conditions under which upward propagation, downward propagation or even propagation failure can occur in both bi-infinite and semi-infinite idealizations of the model. We also study the long-time behavior of the system when either a fixed external input is constantly presented at the first layer of the network or when this external input consists in the presentation of constant input with large amplitude for a fixed time window followed by a reset to a down state of the network for all later times. In both cases, we numerically demonstrate the existence of threshold behavior for the amplitude of the external input characterizing whether or not a full propagation within the network can occur. Our theoretical results are consistent with predictive coding theories and allow us to identify regions of parameters that could be associated with dysfunctional perceptions.
Introduction
In both neuroscience and machine learning, Predictive Coding (PC) has emerged as a dominant theory of brain function. It posits that the brain is a hierarchical inference engine where top-down signals convey predictions about sensory input, and bottom-up signals carry the residual "prediction errors" used to update internal models. While linear PC models have been extensively studied, they fail to capture the essential nonlinearities of biological neurons and modern artificial neural networks. This research bridges that gap by introducing a continuous-time, nonlinear mathematical framework to characterize how information flows through deep, bidirectional hierarchies.
Mathematical Framework and Bistability
The authors derive their model by taking the continuous-time limit of standard discrete-time predictive coding update rules, resulting in a system of functional differential equations. To ensure mathematical tractability, the hierarchy is modeled as a 1D lattice where each layer connects 1-to-1 with its neighbors.
A critical component of the model is the introduction of a sigmoidal activation function. This nonlinearity creates a bistable system where each layer can exist in one of two stable equilibria: a low-activity "down" state (representing rest) or a high-activity "up" state (representing active processing).
As shown in the figure above, for a range of the parameter $\theta$, the system exhibits three equilibria: the stable "up" state $x_u$, the stable "down" state $x_d$, and an unstable intermediate state $x_m$. This bistability is the engine that allows information to propagate as traveling waves, flipping layers from "down" to "up" as a signal passes through.
Traveling Waves and Information Flow
The core of the analysis involves treating the spread of neural activity as a traveling wave moving through a discrete medium. The speed and direction of these waves are primarily governed by a feedback parameter, $q$, which represents the strength of top-down error correction.
- Directionality: High values of $q$ favor top-down propagation, while low values favor bottom-up flow.
- Propagation Failure (Pinning): In specific parameter regimes, the wave speed can drop to zero. This "pinning" effect means a signal becomes trapped at a certain layer and fails to propagate further. Biologically, this may correspond to unconscious perception, where a stimulus is registered by sensory organs but fails to reach higher-order cognitive centers.
- The "Goldilocks" Zone: The study identifies a balanced regime where both bottom-up and top-down propagation are possible, suggesting this represents the optimal state for healthy cognitive function.
Key Results: Thresholds and Dynamic Regions
By mapping the signs of upward and downward wave speeds, the authors partitioned the parameter space into nine distinct behavioral regions. Beyond these steady-state dynamics, the research explored how the network responds to external stimuli:
- Amplitude Thresholds ($s_0^*$): For a constant external stimulus to initiate a full bottom-up wave, its amplitude must exceed a sharp mathematical threshold. Stimuli below this threshold stagnate in the initial layers.
- Duration Thresholds ($\tau^*$): For transient or "flashed" stimuli, there is a critical minimum duration required to trigger a self-sustaining wave. If the stimulus is removed too quickly, the network reverts to its resting state.
Implications and Future Work
These findings have significant implications for understanding psychiatric conditions. Extremes in the parameter space—where either bottom-up or top-down signals dominate—could model the autism-schizophrenia continuum. For instance, hallucinations in schizophrenia might be viewed as uncontrolled, dominating top-down waves that override sensory input.
While the current model assumes simplified 1-to-1 connectivity and permanent "up" states, it provides a rigorous foundation for future work. Subsequent iterations will likely incorporate more complex connectivity patterns and negative feedback mechanisms, such as synaptic depression, to allow the network to return to a resting state naturally after processing.
Summary updated