Abstract
Learning probability distributions on the weights of neural networks (NNs) has recently proven beneficial in many applications. Bayesian methods, such as Stein variational gradient descent (SVGD), offer an elegant framework to reason about NN model uncertainty. However, by assuming independent Gaussian priors for the individual NN weights (as often applied), SVGD does not impose prior knowledge that there is often structural information (dependence) among weights. We propose efficient posterior learning of structural weight uncertainty, within an SVGD framework, by employing matrix variate Gaussian priors on NN parameters. We further investigate the learned structural uncertainty in sequential decision-making problems, including contextual bandits and reinforcement learning. Experiments on several synthetic and real datasets indicate the superiority of our model, compared with state-of-the-art methods.
Introduction
Standard deep learning models typically perform point estimation, seeking a single "best" set of weights. This approach often leads to overconfident predictions and a failure to quantify model uncertainty, which is critical in high-stakes environments. Bayesian Neural Networks (BNNs) mitigate this by learning a distribution over weights. However, traditional BNNs frequently rely on the assumption that weights within a layer are independent (isotropic Gaussian priors). In practice, neural network weights exhibit significant structural correlations. While Stein Variational Gradient Descent (SVGD) provides a non-parametric framework for learning these distributions, it struggles with the computational complexity of modeling high-dimensional weight structures.
Structural Stein Variational Gradient Descent (S2VGD)
To address these limitations, we introduce Structural Stein Variational Gradient Descent (S2VGD). This framework integrates Matrix Variate Gaussian (MVG) priors into the SVGD process, allowing the model to capture intra-layer correlations between rows and columns of the weight matrix.
Layer Decomposition and Householder Flows
Modeling full structural correlations is computationally expensive. S2VGD achieves efficiency through a specific layer decomposition. The weight matrix $W$ is decomposed into a product of five matrices: $W = P \Lambda_1 C \Lambda_2 Q^\top$. Here, $C$ represents the core weights, while $P, Q, \Lambda_1,$ and $\Lambda_2$ capture the structural dependencies.
To represent the orthogonal matrices $P$ and $Q$ without a quadratic parameter explosion, we utilize Householder flows. These are a series of elementary reflections that can represent complex rotations using only a few vectors, drastically reducing the parameter space while maintaining the necessary structural expressiveness.
Experimental Evaluation
We evaluated S2VGD across various tasks, including regression, classification, and sequential decision-making. In regression tasks using UCI datasets, S2VGD consistently achieved lower Root Mean Squared Error (RMSE) and higher log-likelihood compared to existing BNN methods like VMG. On the MNIST classification task, S2VGD reached a test error of 1.30%, outperforming both standard SVGD and traditional SGD.
Sequential Decision-Making
The primary advantage of structural uncertainty is observed in sequential decision-making, where agents must balance exploration and exploitation. By capturing "structural doubt," S2VGD allows agents to explore environments more intelligently.
In experiments on the Yahoo! Today Module dataset—a real-world news recommendation benchmark with 45 million visits—S2VGD demonstrated a significantly higher Click-Through Rate (CTR) than standard SVGD and the classic LinUCB algorithm. Similar improvements were noted in reinforcement learning tasks (e.g., Cartpole and Double Pendulum), where S2VGD converged to optimal rewards faster than its counterparts.
Conclusion
S2VGD demonstrates that capturing intra-layer weight correlations is not just a theoretical improvement but a practical necessity for robust uncertainty estimation. By utilizing Householder transformations and matrix variate priors, we provide a computationally efficient way to model structural uncertainty. This leads to superior performance in complex tasks, particularly in sequential decision-making where accurate uncertainty estimates drive more effective exploration.
Summary updated