Introduction
Preventing overfitting in online learning—where data arrives continuously one point at a time—presents a unique challenge compared to traditional batch learning. This paper addresses this challenge by linking the Extended Kalman Filter (EKF) to Bayesian estimation. By doing so, the authors propose methods to automatically tune the "black box" noise parameters of the Kalman filter, effectively transforming it into a self-regulating optimizer for neural networks.
The Challenge of Sequential Learning
In traditional "batch" machine learning, models have the luxury of processing an entire dataset at once. This allows for techniques like cross-validation or bootstrapping to determine the right model complexity and prevent overfitting (regularisation).
However, in sequential learning (or online learning), data arrives in a continuous stream. Cross-validation is impossible because future data is unavailable, and the underlying system being modeled might be non-stationary (changing over time). Without a mechanism to control model complexity on the fly, sequential estimators like Multi-Layer Perceptrons (MLPs) are prone to instability and poor generalization. The authors specifically target the difficulty of tuning the Extended Kalman Filter (EKF). While the EKF is a powerful tool for sequential tasks, it typically requires researchers to manually "guess" initial noise covariance matrices ($Q$ and $R$) through tedious trial and error.
Proposed Methods
The authors frame the training of a neural network as a state-estimation problem, where the network's weights are treated as the "state" of a dynamical system evolving over time. To improve the EKF for training MLPs, the paper introduces two primary approaches:
- Updating Covariance via Back-Propagation: Drawing inspiration from variable learning rate techniques, the authors approximate the weight covariance matrix ($P$) as a diagonal matrix and update it using error back-propagation. This significantly reduces the computational burden and eliminates the need to manually select the process noise matrix ($Q$).
- The Bayesian Link: The core theoretical contribution is establishing a formal mathematical link between the EKF and David MacKay’s Bayesian "Evidence Framework." The authors demonstrate that the noise matrices in a Kalman filter ($Q$ and $R$) are mathematically related to Bayesian hyperparameters ($\alpha$ and $\beta$) that control the prior distribution of weights and measurement noise. Using a Maximum A Posteriori (MAP) approach, they derive recursive formulas to automatically update these noise matrices at each time step.
Datasets and Evaluation
To evaluate the proposed algorithms, the paper utilizes a synthetic, non-stationary, multivariate time-series process.
- Structure: The data follows a linear relationship for the first 200 time steps and subsequently switches to a complex nonlinear relationship involving sine functions for the remaining 800 steps.
- Noise: Uniformly distributed noise is added to both the inputs and the outputs to rigorously test the robustness of the regularisation techniques.
Standard real-world benchmarks (like MNIST) are bypassed in favor of this synthetic data to focus specifically on the theoretical framework for sequential, non-stationary environments.
Key Insights
- The Kalman Filter as a Second-Order Optimizer: The EKF functions as a more sophisticated version of back-propagation. While back-propagation relies on first-order derivatives, the EKF uses second-order statistics (covariances), allowing for more principled "error bars" on predictions.
- Noise is Regularisation: A critical realization in this work is that the process noise ($Q$) and measurement noise ($R$) in control theory are functionally equivalent to regularisation coefficients in machine learning. Tuning the filter's "noise" directly controls the model's complexity.
- Automating the "Black Art": By linking the EKF to Bayesian methods, the authors provide a pathway away from heuristic guessing, enabling self-tuning autonomous learners.
Key Results
The authors compared four distinct methods: Conventional EKF, EKF with back-prop updates, EKF with Bayesian MAP updates, and EKF with the Evidence framework.
Interestingly, the conventional EKF (with hand-tuned parameters) slightly outperformed the automated Bayesian methods on the toy problem. The authors attribute this to the simplicity of the problem, where a human can effectively "guess" the optimal noise values. However, the automated Bayesian methods performed competitively without requiring any manual tuning. This proves they are highly viable for complex, real-world scenarios where optimal parameters are not obvious.
Limitations and Future Work
- Computational Complexity: The EKF is computationally expensive, scaling at $O(mq^2)$ where $q$ is the number of weights. This is a significant bottleneck for large modern neural networks. The authors suggest "decoupling" weights to mitigate this.
- Initial Conditions: While the framework automates $Q$ and $R$, the filter remains sensitive to the initial conditions ($w_0, P_0$). A proposed workaround is using a small batch of initial data to "warm up" the model using standard back-propagation.
- Scalability: Because the toy problem is low-dimensional, future work must address how these Bayesian updates scale to much larger architectures and more volatile non-stationary environments.
Summary updated