Random-Set Convolutional Neural Network (RS-CNN) for Epistemic Deep Learning

Shireen Kudukkil Manchingal, Muhammad Mubashar, Kaizheng Wang, Keivan Shariatmadar, Fabio Cuzzolin

TL;DR

Random-Set Neural Networks (RS-NN) move beyond standard point-probability predictions by predicting belief functions over sets of classes. This allows models to explicitly represent epistemic uncertainty (ignorance due to limited data), resulting in state-of-the-art accuracy, superior out-of-distribution detection, and remarkable robustness to adversarial attacks—all while remaining scalable to large architectures like Vision Transformers.

What Problem Does This Paper Address?

Deep learning models are notoriously overconfident. Even when presented with "Out-of-Distribution" (OoD) data—samples completely unlike anything in their training set—standard neural networks will often assign a high-probability score to a random class. This is a failure to distinguish between aleatoric uncertainty (inherent noise in the data, like a blurry image of a cat) and epistemic uncertainty (the model’s lack of knowledge, like seeing a platypus for the first time).

Existing solutions like Bayesian Neural Networks (BNNs) or Deep Ensembles attempt to solve this but often come with massive computational overhead, difficulty in scaling to large datasets like ImageNet, or a reliance on sensitive "prior" distributions that can bias the results. The authors of this paper seek a "wrapper" approach that provides reliable uncertainty estimates without sacrificing speed or scalability.

Methods

The core innovation is the Random-Set Neural Network (RS-NN). Instead of a standard softmax layer that outputs a probability vector for $N$ individual classes, RS-NN predicts values for a collection of sets of classes.

  1. Belief Functions & Random Sets: Drawing from the Dempster-Shafer theory of evidence, the model assigns "mass" to subsets of classes. For example, if a model is unsure if an image is a "car" or a "truck," it can assign mass to the set {car, truck} directly, rather than being forced to split its confidence between them.
  2. Budgeting (The Scalability Hack): Predicting every possible subset of classes is computationally impossible (there are $2^N$ subsets). To fix this, the authors use a "budgeting" method: they cluster features from the training data using Gaussian Mixture Models (GMMs) to identify which classes naturally overlap. They then only predict belief values for the $N$ original classes plus the $K$ most relevant overlapping sets.
  3. Loss Function: The network is trained using a modified Binary Cross-Entropy (BCE) loss that encourages the model to predict high belief values for any set containing the true class. They also include regularization terms to ensure the predicted "masses" are non-negative and sum to one.
  4. Inference: To make a final decision, the model calculates the pignistic probability—essentially the center of mass of the predicted belief function. Uncertainty is measured by the "width" of the credal set (the range of possible probabilities consistent with the evidence).

Datasets

The authors conducted an extensive evaluation across multiple domains:

  • Image Classification (In-Distribution): MNIST, CIFAR-10, CIFAR-100, Intel Image, and the massive ImageNet-1K.
  • Out-of-Distribution (OoD) Benchmarks: SVHN, Fashion-MNIST, K-MNIST, and ImageNet-O (a dataset specifically designed to fool ImageNet models).
  • Text Classification: The BBC Text dataset (In-Distribution) vs. an Emotion Detection dataset (OoD).

Key Insights and Ideas

  • Sets Capture Ignorance Better Than Points: By allowing the model to say "It's one of these three classes, but I don't know which," RS-NN avoids the "forced choice" problem of softmax, which naturally leads to better-calibrated uncertainty.
  • The "Wrapper" Advantage: RS-NN isn't a new architecture; it's a new way to handle the output of an architecture. The authors successfully applied it to ResNet50, WideResNet, VGG16, Inception V3, and even Vision Transformers (ViT).
  • Adversarial Immunity: One of the most striking findings is that RS-NN is naturally resistant to adversarial attacks like FGSM and PGD. Because these attacks rely on pushing a model toward a specific wrong class, the RS-NN often responds by simply increasing its uncertainty (widening the credal set) rather than committing to the wrong answer.
  • Efficiency: Unlike Ensembles (which require running $M$ models) or BNNs (which require sampling), RS-NN has an inference time nearly identical to a standard CNN.

Key Results

  • Accuracy: RS-NN consistently outperformed state-of-the-art BNNs (like LB-BNN and FSVI) and Ensembles. On CIFAR-10, it achieved 93.53% accuracy compared to the standard CNN's 92.08%. On ImageNet, it reached 79.92% (Top-1).
  • OoD Detection: RS-NN showed a massive lead in detecting unfamiliar data. On the CIFAR-10 vs. Intel Image benchmark, it achieved an AUROC of 97.39, significantly higher than BNNs (~82) or standard CNNs (~87).
  • Calibration: It achieved the lowest Expected Calibration Error (ECE) across almost all tests, meaning its "confidence" scores actually matched its real-world accuracy.
  • Adversarial Robustness: Under a strong PGD attack on CIFAR-10, a standard CNN's accuracy dropped to near 0%, while RS-NN maintained nearly 60% accuracy.

Limitations and Future Work

  • Budgeting Overhead: While the budgeting step is a one-time cost, it currently requires a pre-trained feature extractor and a clustering phase (taking about 60 minutes for ImageNet).
  • Hyperparameter Tuning: The number of focal sets ($K$) and the regularization weights ($\alpha, \beta$) are currently manual hyperparameters. The authors suggest a dynamic strategy for adjusting $K$ based on class overlap in the future.
  • Beyond Classification: The current work is focused on classification. The authors note that extending this random-set approach to regression tasks is a key next step.
  • Mass Regularization: Future iterations may look for ways to guarantee valid belief functions (non-negative masses summing to 1) without needing explicit penalty terms in the loss function.

Summary updated

Abstract

Machine learning is increasingly deployed in safety-critical domains where robustness against adversarial attacks is crucial and erroneous predictions could lead to potentially catastrophic consequences. This highlights the need for learning systems to be equipped with the means to determine a model's confidence in its prediction and the epistemic uncertainty associated with it, 'to know when a model does not know'. In this paper, we propose a novel Random-Set Convolutional Neural Network (RS-CNN) for classification which predicts belief functions rather than probability vectors over the set of classes, using the mathematics of random sets, i.e., distributions over the power set of the sample space. Based on the epistemic deep learning approach, random-set models are capable of representing the 'epistemic' uncertainty induced in machine learning by limited training sets. We estimate epistemic uncertainty by approximating the size of credal sets associated with the predicted belief functions, and experimentally demonstrate how our approach outperforms competing uncertainty-aware approaches in a classical evaluation setting. The performance of RS-CNN is best demonstrated on OOD samples where it manages to capture the true prediction while standard CNNs fail.
BibTeX
@inproceedings{manchingal2024random,
  author = {Shireen Kudukkil Manchingal and Muhammad Mubashar and Kaizheng Wang and Keivan Shariatmadar and Fabio Cuzzolin},
  title = {{Random-Set Convolutional Neural Network (RS-CNN) for Epistemic Deep Learning}},
  booktitle = {KU Leuven},
  year = {2024},
  url = {https://openreview.net/forum?id=G02odGoR5Z},
}

Loading related content...

Figure