Single Phase Learning in Continuous Time Neural Networks

This direction is merged into another direction: weight transport problem. Its last version stays readable and citable here.

Version 1 of 151 papers63 conceptsfrozen 2026-10-04

In modern deep learning, training follows a strict "stop-and-go" rhythm. To update a network, we first perform a forward pass to generate a prediction, "freeze" the activations, and then perform a backward pass to distribute error signals. This temporal separation is the cornerstone of backpropagation, but it creates a fundamental disconnect with biology.

The brain is a continuous-time dynamical system. It does not have a global clock to synchronize phases, nor does it pause its sensory processing to wait for a "teaching signal" to arrive from the future. As noted by Matthias Zöhrer and Franz Pernkopf, natural systems learn to represent and categorize information simultaneously. This research direction focuses on single-phase learning: mechanisms that allow neural networks to perform credit assignment in real-time, without alternating between distinct inference and training stages.

The Bottleneck of Two-Phase Learning

Most "biologically plausible" alternatives to backpropagation still struggle with temporal locality. For instance, Contrastive Hebbian Learning and Equilibrium Propagation typically require a "free phase" (processing input) and a "nudged phase" (adjusting toward a target).

As Rasmus Høier points out, this requirement makes these algorithms orders of magnitude slower on digital hardware and nearly impossible to implement in decentralized biological circuits. If a neuron must "remember" its state from the free phase to compare it with the nudged phase, it requires complex internal storage buffers that biological neurons likely lack. Geoffrey E. Hinton and Andrew D. Brown highlighted this early on: biological systems cannot easily perform a "negative phase" while simultaneously processing a continuous stream of new sensory data.

Multiplexing Signals in Dendritic Compartments

One solution to the phase problem lies in the physical structure of the neuron itself. Rather than separating learning into different times, the brain might separate it into different places.

Changze Lv and colleagues propose Dendritic Localized Learning (DLL), which mimics the three-compartment structure of pyramidal neurons. By segregating feedforward inputs and feedback errors into different dendritic branches, the neuron can calculate local prediction errors without needing to switch modes. Similarly, Will Greedy and Rui Ponte Costa introduced BurstCCN, which uses neural bursting to multiplex forward and backward signals. In this model, the timing and frequency of spikes allow the network to "hide" the error signal within the same stream of activity used for inference.

Learning from Transients and Latent Equilibrium

In traditional models, we wait for a recurrent network to "settle" into a steady state before calculating a gradient. This "slowness" is a bottleneck for both biology and neuromorphic chips.

Paul Haider, Mihai A. Petrovici, and their team proposed Latent Equilibrium, a framework where learning occurs during the transient state—the period while the neuron is still changing. By using predictive signals that remain valid even before the network reaches stability, they demonstrate that computation can be "arbitrarily fast" even with slow physical neurons. This eliminates the need to pause input or wait for relaxation, allowing the synaptic weights to update continuously as the data flows through.

Always-On Dynamics and Stochastic Phases

If we cannot eliminate phases entirely, can we at least make them "messy" and asynchronous? Ezekiel Williams and Guillaume Lajoie have shown that contrastive learning can work with "always-on" updates. Instead of rigid, periodic cycles, they use stochastic phase selection.

The intuition here is a separation of timescales: if the internal neural dynamics are fast and the learning rate is slow, the weights effectively "average" over the network's fluctuations. As long as there is some temporal overlap between the input and the error signal—a principle explored by Marc Gong Bacvanski and Tomaso Poggio—the network can self-organize.

Toward Collectionless AI

The shift toward single-phase learning mirrors a broader move toward Collectionless Artificial Intelligence. As Marco Gori argues, biological organisms do not shuffle data into batches or revisit past experiences in a static buffer. They process a one-way, continuous stream of information.

By removing the requirement for global synchronization and discrete training phases, researchers are moving toward AI that is not just "biologically inspired" in its architecture, but biologically plausible in its existence—learning from the world in real-time, one spike at a time.

Papers in this version

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_single_phase_learning_in_continuous_time_8d73434c,
  title = {Single Phase Learning in Continuous Time Neural Networks},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/single-phase-learning-in-continuous-time-neural-networks/rdr_8d73434c04304732b4f43755df6300a6},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_single_phase_learning_in_continuous_time_8d73434c_v1,
  title = {Single Phase Learning in Continuous Time Neural Networks},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/single-phase-learning-in-continuous-time-neural-networks/rdr_8d73434c04304732b4f43755df6300a6/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}