SoU-RHS: Bayesian Structured Sparsity for Compact Architecture Recovery in CNNs

Rafael Mouallem Rosa, Julyan ArbelarXiv 2026 · arXiv

TL;DR

SoU-RHS is a novel Bayesian approach for pruning convolutional neural networks (CNNs) that moves beyond simple compression to true "architecture recovery." By combining horseshoe shrinkage with a specialized uncertainty-driven mechanism, it identifies compact, stable architectures that maintain high accuracy while providing significantly better uncertainty estimates than standard deterministic methods.

What Problem Does This Paper Address?

Modern deep learning often relies on "overparameterized" models—networks with far more parameters than strictly necessary for the task. While this helps training, it leads to massive costs in memory and computation during deployment.

Researchers use structured sparsity (pruning entire filters or channels) to create efficient models. However, existing methods often suffer from two main issues:

  1. Instability: The final architecture can change wildly depending on the initial model size or the specific pruning threshold chosen.
  2. Poor Uncertainty: Most pruning methods are deterministic and don't tell you how confident the model is in its predictions, which is a major drawback for safety-critical applications.

The authors of this paper aim to find a "stable" way to recover the underlying compact structure of a network while preserving the probabilistic benefits of Bayesian inference.

Methods

The authors introduce SoU-RHS, which stands for Sources of Uncertainty Regularized Horseshoe Shrinkage. It builds on two key components:

  • Regularized Horseshoe Shrinkage (RHS): This is a Bayesian prior designed for sparsity. It assumes most parameters should be zero (the "horseshoe" shape) but allows a few important ones to remain large. In this paper, it's applied at the filter level rather than the weight level.
  • Sources of Uncertainty Sparse Global-Local (SoU-SGL): This is the "secret sauce." Standard Bayesian shrinkage often leaves "inactive" filters with very small, non-zero values, making it hard to decide what to prune. SoU-SGL treats the uncertainty of these local scales as an optimization problem. By driving the confidence in these scales toward zero, it "sharpens" the decision, forcing inactive filters much closer to zero while letting active ones thrive.

Essentially, the method adopts an "optimize structure, infer parameters" philosophy. It uses optimization-like behavior to decide which filters to keep and standard Bayesian inference to learn the values of those filters.

Datasets

The method was evaluated on four standard image-classification benchmarks:

  • CIFAR-10 & CIFAR-100: 32x32 natural images with 10 and 100 classes.
  • SVHN: 32x32 real-world house number digits.
  • Tiny-ImageNet: A more challenging 64x64 dataset with 200 classes.

The authors used a VGG-style backbone for all experiments, starting with different "overparameterized" widths (128, 256, and 512 filters per layer) to test the stability of the recovery process.

Key Insights and Ideas

  • Stability is the New Metric: A key finding is that SoU-RHS is remarkably stable. Whether you start with a wide or narrow model, it tends to recover a similar final architecture for a given dataset. This suggests the method is finding an "intrinsic" structure adapted to the data, rather than just compressing whatever it was given.
  • Sharper Pruning Decisions: Unlike standard Horseshoe priors, which can be sensitive to the pruning threshold, SoU-RHS creates a clear "gap" between active and inactive filters. This makes the choice of threshold much less critical.
  • The Architecture-Inference Interaction: Interestingly, when the authors retrained models from scratch on the recovered architectures, the Bayesian models (SoU-RHS) stayed strong or even improved, while deterministic baselines often saw performance degradation. This suggests that the architectures recovered by SoU-RHS are particularly well-suited for probabilistic inference.
Figure 1: Comparison of active filters across thresholds for Tiny ImageNet. Note how SoU-RHS (blue) remains stable across a wide range of thresholds compared to standard RHS (orange).

Key Results

  • Superior Compactness: SoU-RHS consistently recovered smaller architectures than deterministic baselines like Network Slimming (SLIM) and Structured Sparsity Learning (SSL), often by a significant margin.
  • Better Probabilistic Performance: In every single configuration tested, SoU-RHS achieved a lower Negative Log-Likelihood (NLL) than the baselines. It also generally achieved lower Expected Calibration Error (ECE), meaning its predicted probabilities are more trustworthy.
  • Competitive Accuracy: Despite being much smaller, SoU-RHS models maintained accuracy competitive with (and sometimes exceeding) dense, unpruned models, especially on more challenging tasks like Tiny-ImageNet.

Limitations and Future Work

The authors acknowledge a few areas for improvement:

  • Training Cost: Bayesian models typically require longer training schedules than deterministic ones. SoU-RHS is no exception, incurring a higher initial training cost to allow the architecture to stabilize.
  • Scope of Evaluation: The current study is limited to VGG-style CNNs and medium-scale datasets. Future work is needed to see how well this "stability" property holds up in massive Transformer models or on datasets like full ImageNet.
  • Inference Speed: While the architecture is compact, Bayesian inference requires multiple forward passes (Monte Carlo sampling) to get uncertainty estimates, which can offset some of the speed gains from pruning. The authors explore this trade-off in the appendix, showing that even with a few samples, the probabilistic gains are substantial.

Summary updated

Abstract

Highly parameterized convolutional neural networks often achieve strong predictive performance, but their memory, computation, and deployment costs motivate the search for methods that recover compact architectures from large reference models. We propose SoU-RHS, a Bayesian structured-sparsity procedure that augments Regularized Horseshoe Shrinkage (RHS) with the so-called Sources of Uncertainty Sparse Global-Local mechanism (SoU-SGL). The resulting model sharpens the separation between active and inactive filters, providing an effective mechanism for architecture recovery. A key property of SoU-RHS is stability: across different overparameterized starting widths and under a reasonable range of pruning thresholds, it recovers compact architectures with similar total size and layer-wise allocation. This supports the interpretation that the procedure is identifying a compact structure adapted to the data, rather than merely compressing each initial model separately. Across standard image-classification benchmarks, SoU-RHS recovers substantially smaller architectures than deterministic structured-sparsity baselines, while maintaining competitive accuracy, achieving lower negative log-likelihood in every evaluated configuration and generally lower expected calibration error.
BibTeX
@misc{rosa2026sou,
  author = {Rafael Mouallem Rosa and Julyan Arbel},
  title = {{SoU-RHS: Bayesian Structured Sparsity for Compact Architecture Recovery in CNNs}},
  year = {2026},
  eprint = {2611.24166},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2611.24166},
}

Loading related content...

Figure