Input Driven Suppression of Chaos in Recurrent Networks

This direction is split into other directions: variability quenching. Its last version stays readable and citable here.

Version 1 of 176 papers139 conceptsfrozen 2026-10-04

Recurrent neural networks (RNNs) are inherently volatile. When internal connections are strong enough to store complex memories, they often cross a threshold into chaos—a state where tiny perturbations grow exponentially, rendering the network’s output unpredictable. Traditionally, researchers avoided this by strictly limiting the "spectral radius" of connection weights to keep the system stable.

However, a growing body of research suggests that this "autonomous" view of stability is incomplete. In practice, neural networks are rarely left to their own devices; they are constantly driven by external inputs. Researchers have discovered that these external signals—and even stochastic noise—can dynamically suppress internal chaos, allowing networks to operate in high-performance regimes that were previously thought to be unstable.

The Stabilizing Force of External Stimuli

The core intuition behind this research direction is that external inputs "slave" the network's internal dynamics. In an autonomous state (no input), a recurrent network might wander through a high-dimensional chaotic landscape. But as Kanaka Rajan and Haim Sompolinsky demonstrated in Interactions between Intrinsic and Stimulus-Evoked Activity in Recurrent Neural Networks, an external stimulus can nonlinearly suppress these chaotic fluctuations. The input forces the network’s trajectories to collapse into stable, stimulus-locked states, effectively "quenching" the internal noise.

This isn't just a theoretical curiosity; it is a fundamental feature of biological brains. Byron M. Yu and Mark M. Churchland observed in Neural Variability in Premotor Cortex Provides a Signature of Motor Preparation that neural variability drops significantly the moment a subject prepares to move. This "settling" into a consistent state suggests that the brain uses task-relevant signals to stabilize its own chaotic circuitry.

Why Input Suppresses Chaos

Why does an external signal make a chaotic system well-behaved? There are two primary mechanisms identified across the literature:

  1. Saturation and Gain Control: In networks with saturating activation functions (like tanh or sigmoid), strong inputs push neurons into the "flat" regions of their response curves. As Jonathan Dong and colleagues show in Asymptotic Stability in Reservoir Computing, this reduces the system's sensitivity to small perturbations. By effectively lowering the "gain" of the neurons, the input shifts the stability boundary, allowing the network to remain non-chaotic even with very strong internal weights.
  2. Attractor-Mediated Stabilization: External inputs can reshape the dynamical landscape itself. Takuya Ito and Markus Siegel argue in Task-evoked activity quenches neural correlations and variability across cortical areas that task-evoked activity pushes the system toward stable fixed-point attractors. This increases the "mathematical stiffness" of the state, making it harder for internal noise to knock the system off course.

Moving Beyond Static Stability Measures

Because the input signal is so influential, traditional metrics like the spectral radius of the weight matrix are often misleadingly conservative. Claudio Gallicchio found in Chasing the Echo State Property that Echo State Networks can remain stable and functional even when their weights far exceed traditional theoretical bounds, provided the input signal is strong enough to act as a regularizing force.

To capture this, researchers like Ryan Vogt and Guillaume Lajoie have turned to Stochastic Lyapunov Exponents. As detailed in On Lyapunov Exponents for RNNs, this approach treats the RNN as a non-autonomous system. Instead of looking at the weights in isolation, it tracks how the hidden state's Jacobian evolves in response to a specific data distribution. This reveals a "locally expansive but globally stable" regime where the network can amplify signals for processing without exploding into chaos.

The Paradox of Stochastic Stabilization

Perhaps the most counter-intuitive finding in this field is that noise itself can be a stabilizer. In deterministic systems with feedback delays, neurons often fall into pathological, synchronized oscillations. Camille E. Rullán Buxó and Jonathan W. Pillow demonstrated in Poisson balanced spiking networks that introducing probabilistic spiking (noise) prevents this "ping-ponging" effect. The noise introduces temporal jitter that smooths the population's average gain, preventing catastrophic synchronization.

Similarly, Soon Hoe Lim and Michael W. Mahoney showed in Noisy Recurrent Neural Networks that treating hidden state transitions as stochastic differential equations can keep trajectories bounded. In this view, noise isn't just interference to be filtered out; it is a tool that ensures the network remains in a predictable, non-divergent regime over long sequences.

Why It Matters: The Edge of Chaos

The goal of tuning these dynamics is often to reach the "edge of chaos"—the transition point between rigid stability and fluid chaos. Nils Bertschinger and Thomas Natschläger famously argued in At the Edge of Chaos that this is where RNNs achieve peak computational power. By using inputs and noise to dynamically suppress chaos, we can push networks into this critical regime, optimizing the balance between the sensitivity needed to process new information and the stability needed to remember the past.

Papers in this version

Showing 60 of 76 papers.

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_input_driven_suppression_of_chaos_in_rec_7c77f2de,
  title = {Input Driven Suppression of Chaos in Recurrent Networks},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/input-driven-suppression-of-chaos-in-recurrent-networks/rdr_7c77f2de82384c6eb4281a4befb1a22e},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-19},
}

This exact version (version 1, frozen text):

@misc{lacuna_input_driven_suppression_of_chaos_in_rec_7c77f2de_v1,
  title = {Input Driven Suppression of Chaos in Recurrent Networks},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/input-driven-suppression-of-chaos-in-recurrent-networks/rdr_7c77f2de82384c6eb4281a4befb1a22e/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-19},
}