In standard machine learning, we typically train models under the assumption of a static, fully accessible dataset. However, real-world data often arrives in continuous streams or discrete sequential tasks. When a model is naively fine-tuned on new data, it suffers from catastrophic forgetting—overwriting previously learned patterns to accommodate the new ones.
The most mathematically elegant solution to this problem stems from a fundamental tenet of Bayesian inference: today's posterior is tomorrow's prior. By treating learning as a recursive process, we can compress everything a model has learned from historical data into a probability distribution, and use that distribution as the starting point for learning from new data.
The Core Mechanism: Posterior-to-Prior Substitution
The intuition behind recursive Bayesian updating is straightforward. Suppose a model encounters a sequence of tasks or data batches. Instead of maintaining fixed point-estimates for the model's parameters (such as the weights of a neural network), we maintain a probability distribution over them.
As Ole Winther et al. demonstrated in Bayesian online learning in the perceptron, online learning can be framed as a recursive process where information from each new training example is compressed directly into the parameters of an approximate posterior distribution. Once the posterior is calculated for the current batch of data, the individual data points can be safely discarded.
When the next batch of data arrives, the previous posterior distribution is substituted directly as the prior for the new update step. This mechanism acts as a "mathematical memory." As Jeevan Thapa et al. note in Bayesian Adaptation of Network Depth and Width for Continual Learning, this recursive loop protects previously acquired information from being overwritten during new task acquisition without requiring access to the original raw data.
[ Prior P(θ) ] ──> [ Learn Task 1 ] ──> [ Posterior P(θ | Task 1) ]
│
▼
[ Posterior P(θ | Task 1) ] ──────────────────> [ New Prior P(θ) ]
│
▼
[ New Prior P(θ) ] ──> [ Learn Task 2 ] ──> [ Posterior P(θ | Task 1, 2) ]
Why It Works: Uncertainty as a Regularizer
To understand why this prevents catastrophic forgetting, we must look at how parameter variance behaves during the update cycle.
When a model learns a task, it doesn't just learn the optimal values for its weights; it also learns how sensitive the task performance is to changes in those weights. If a specific weight is critical to solving Task 1, the model's uncertainty about that weight shrinks, resulting in a narrow posterior variance. Conversely, if a weight is irrelevant to Task 1, its variance remains wide.
When the posterior becomes the prior for Task 2, these variances act as directional regularizers. As shown by Max Kochurov et al. in Bayesian Incremental Learning for Deep Neural Networks and Cuong V. Nguyen et al. in Variational Continual Learning, the recursive update penalizes changes to parameters that the model is highly certain about. The optimization landscape for Task 2 is effectively constrained: the model is forced to find solutions within the wide-variance regions of the parameter space, leaving the critical, narrow-variance parameters intact.
Approximating the Intractable
While exact Bayesian updating is clean in theory, it is computationally intractable for deep neural networks because calculating the true posterior requires integrating over millions of parameters. Modern research focuses on different approximation strategies to make this recursive loop viable.
Weight-Space Approximations
One approach is to approximate the geometry of the weight posterior using local curvature. For instance, Hippolyt Ritter et al. in Online Structured Laplace Approximations for Overcoming Catastrophic Forgetting utilize a Kronecker-factored Laplace approximation. By capturing the interactions between weights within the same layer, they construct a quadratic penalty that prevents parameters from drifting away from configurations that were optimal for earlier tasks.
Function-Space Approximations
An alternative paradigm argues that regularizing weights is too restrictive because many different weight configurations can yield the same input-output behavior. Instead of regularizing the parameter space, Tim G. J. Rudner et al. introduce Continual Learning via Sequential Function-Space Variational Inference. In this framework, the recursive update happens in the prediction space. The prior is the predictive distribution over functions from the previous task, and the objective minimizes the divergence between the new predictive function and the old one, allowing internal parameters to shift flexibly as long as the functional mapping remains stable.
The Plasticity Dilemma: Preventing Variance Collapse
A fundamental challenge in recursive Bayesian updating is managing the trade-off between stability and plasticity. As a model processes more data over time, its parameter distributions naturally narrow. If a model becomes too certain, the posterior variance collapses toward zero. When this happens, the prior becomes an unyielding regularizer, rendering the model rigid and incapable of learning new information.
To sustain the model's ability to adapt in streaming or non-stationary environments, researchers introduce mechanisms to artificially inject uncertainty or modulate learning rates based on environmental changes:
- Transition Models: Ha Nguyen et al. propose a solution in Adaptive infinite dropout for noisy and sparse data streams by incorporating a transition model between recursive updates. This model allows global parameters to fluctuate slightly between mini-batches, preventing the posterior variance from collapsing and ensuring the network remains sensitive to new data streams.
- Surprise-Modulated Learning: Another approach relies on detecting environmental shifts. Vasiliki Liakoni et al. in Learning in Volatile Environments With the Bayes Factor Surprise demonstrate that updates can be modulated by a surprise signal. By comparing how well new observations fit the current belief versus a naive baseline prior, the model can dynamically scale back its certainty when it encounters a sudden change in the data distribution, restoring its plasticity on the fly.
By treating online learning not as a sequence of isolated optimization problems, but as a continuous, recursive refinement of beliefs, recursive posterior updating provides a mathematically grounded framework for lifelong learning.
Key Foundational Work
Modeling Temporal Structure in Classical Conditioning (2001) — contributed 1 concept elements to this direction.
Bayesian online learning in the perceptron (1997) — contributed 1 concept elements to this direction.