K-prop: Deep Online Learning by Backpropagating Temporal Kernels

Bingkun Liu, Paul Haider, Federico Benitez, Mihai A. Petrovici, Walter SennarXiv 2026 · arXiv

TL;DR

K-prop is a novel online learning algorithm for deep leaky neural networks that slashes the computational cost of exact online learning (RTRL) from a prohibitive $O(n^4)$ to a practical $O(n^2)$. By backpropagating "kernel gains" instead of full gradient trajectories, it achieves performance nearly identical to offline backpropagation (BPTT) while remaining fully compatible with real-time, streaming data and analog hardware.

What Problem Does This Paper Address?

Training neural networks to process temporal data—like speech, video, or sensor streams—usually relies on Backpropagation Through Time (BPTT). While effective, BPTT is "offline": it requires storing the entire history of a sequence and replaying it backward. This is a dealbreaker for real-time systems, biological brains, and analog "neuromorphic" hardware that must learn on the fly without massive memory buffers.

The alternative, Real-Time Recurrent Learning (RTRL), is truly online but suffers from a "combinatorial explosion." For a network with $n$ neurons, RTRL requires $O(n^4)$ operations per time step, making it unusable for anything but the smallest toy models. Previous attempts to approximate RTRL often sacrifice gradient accuracy or fail to scale to deep architectures. The authors seek a middle ground: an algorithm that is efficient, deep-scalable, and accurate.

Methods

The researchers introduce Kernel Propagation (K-prop), which focuses on networks of leaky neurons. These neurons act as low-pass filters, naturally creating "temporal kernels" that retain information over time.

K-prop simplifies the gradient calculation in three clever steps:

  1. Separating Nonlinearity from Filtering: It assumes the derivative of the activation function ($\rho'$) changes slowly compared to the neuron's integration time. This allows the math to "move" the nonlinearity outside the temporal filters.
  2. Collapsing Nested Filters: The authors prove a mathematical lemma showing that complex, hierarchical filters (filters of filters) can be rewritten as a simple linear combination of basic exponential kernels.
  3. Propagating Kernel Gains: Instead of tracking every possible path through time and space, K-prop groups these paths by their shared time constants ($\tau$). It then propagates the "gains" ($K$) of these kernels backward through the layers, much like standard backpropagation.
Figure 2: Kernel propagation operation. Example propagation from layer l with 3 neurons to layer l −1 with 4 neurons. The gray box works as a bracket.

Datasets

The authors evaluated K-prop across a diverse set of temporal challenges:

  • N-sum Task: A continuous bit-stream processing task requiring the model to remember and sum the last $N$ bits.
  • MNIST1D: A one-dimensional, temporal sequence version of the classic handwritten digit dataset.
  • Google Speech Commands (GSC): A real-world keyword spotting task (20 commands) requiring long-range temporal dependencies.
  • Reinforcement Learning (RL): Partially observable versions of CartPole and LunarLander, where the agent must infer hidden states (like velocity) from historical data.

Key Insights and Ideas

  • The Power of Decomposition: The "aha!" moment is realizing that the messy, high-dimensional gradient of a deep leaky network can be decomposed into a small set of basis kernels. This turns a search through an exponential number of paths into a sum over a few distinct time constants.
  • Layer-wise Locality: Unlike many other online methods, K-prop remains "layer-wise local." This means it can easily incorporate modern deep learning staples like Batch Normalization, Dropout, and Residual connections.
  • Exploiting Diagonal Dynamics: Most RTRL approximations try to reduce complexity by assuming the weight matrix is "low-rank." K-prop instead exploits the fact that leaky integration is "diagonal" (each neuron integrates its own history), which is a much more accurate reflection of how these networks actually work.
Figure 1: Kernel propagation. (a) Calculating gradients involves hierarchical filters along paths of neuron connections. (b) Illustration of kernel decomposition. The complicated temporal structure in the true gradient can be approximated by a sum over ζ weighted by τ-dependent coefficients K. Every ζ (solid) is weighted by corresponding K as a component (dashed in the same color). The sum of all components aligns well with the exact term computed by RTRL.

Key Results

  • Massive Speedup: In the CartPole task, K-prop reduced RTRL’s memory cost by 28x and computational cost by 1,684x, turning an impossible task into a practical one.
  • Competitive Accuracy: On MNIST1D, a 5-layer K-prop network achieved 97.56% accuracy, nearly matching the 97.69% of the "gold standard" BPTT and outperforming other online methods like OSTL (97.03%).
  • High Gradient Quality: The authors measured the "cosine dissimilarity" between K-prop gradients and exact BPTT gradients. The median error remained remarkably low ($< 10^{-2}$), proving the approximation is highly reliable.
  • Robustness: K-prop maintained high performance even when significant Gaussian noise was injected into the learning process, a vital feature for implementation on noisy analog hardware.

Limitations and Future Work

The primary limitation is the slow-varying assumption. K-prop assumes the activation derivative $\rho'$ doesn't change too fast. While this holds for smooth functions like Sigmoid or Tanh, it becomes problematic for Spiking Neural Networks (SNNs), where derivatives are impulsive and discontinuous.

The authors suggest that future work should explore more refined "surrogate derivatives" to bridge this gap. Additionally, while the current version assumes distinct time constants to avoid mathematical singularities, extending the framework to handle identical time constants more elegantly is a logical next step.

Summary updated

Abstract

Self-recurrence represents an important component for temporal processing in neural networks, both artificial and biological, where it appears naturally in the form of leaky integration. However, due to the complex temporal dependencies of the gradient, an efficient and effective method of online training is still lacking. Here we propose K-prop, a novel online learning method for leaky neural networks. It dramatically reduces the computational and memory cost of real time recurrent learning (RTRL) by propagating kernel gains. K-prop also outperforms existing RTRL approximations and achieves performance comparable to backpropagation through time in sequence learning, classification, and reinforcement learning tasks. We further analyze how its sole approximation affects gradient quality and identify conditions under which K-prop remains accurate and effective.
BibTeX
@misc{liu2026k,
  author = {Bingkun Liu and Paul Haider and Federico Benitez and Mihai A. Petrovici and Walter Senn},
  title = {{K-prop: Deep Online Learning by Backpropagating Temporal Kernels}},
  year = {2026},
  eprint = {2611.12530},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.12530},
}
Figure