Distributional Gaussian Processes Layers for Out-of-Distribution Detection

Sebastian Popescu, David Sharp, James H. Cole, Konstantinos Kamnitsas, Ben GlockerThe Journal of Machine Learning for Biomedical Imaging 2022 · DOI

Abstract

Machine learning models deployed on medical imaging tasks must be equipped with out-of-distribution detection capabilities in order to avoid erroneous predictions. It is unsure whether out-of-distribution detection models reliant on deep neural networks are suitable for detecting domain shifts in medical imaging. Gaussian Processes can reliably separate in-distribution data points from out-of-distribution data points via their mathematical construction. Hence, we propose a parameter efficient Bayesian layer for hierarchical convolutional Gaussian Processes that incorporates Gaussian Processes operating in Wasserstein-2 space to reliably propagate uncertainty. This directly replaces convolving Gaussian Processes with a distance-preserving affine operator on distributions. Our experiments on brain tissue-segmentation show that the resulting architecture approaches the performance of well-established deterministic segmentation algorithms (U-Net), which has not been achieved with previous hierarchical Gaussian Processes. Moreover, by applying the same segmentation model to out-of-distribution data (i.e., images with pathology such as brain tumors), we show that our uncertainty estimates result in out-of-distribution detection that outperforms the capabilities of previous Bayesian networks and reconstruction-based approaches that learn normative distributions.

Introduction: The Challenge of Overconfidence

Deep learning models, particularly U-Nets, are the current gold standard for medical image segmentation. However, these models often exhibit a dangerous "blind spot": they tend to be overconfident when encountering data outside their training distribution. In clinical settings, a model trained on healthy brain scans might attempt to segment a scan containing a large tumor as healthy tissue, rather than flagging it as an anomaly.

This failure in Out-of-Distribution (OOD) detection stems from a lack of "distance awareness." While Bayesian Neural Networks (BNNs) attempt to solve this by estimating uncertainty, they often suffer from "feature collapse," where the unique signatures of outliers are lost as data passes through multiple layers. For reliable OOD detection, a model must recognize how far a test point lies from the training manifold at every stage of the network.

Methodology: Distributional Gaussian Process (DistGP) Layers

To address these issues, we propose the Measure-preserving DistGP layer as a drop-in replacement for standard convolutional layers. The architecture relies on several key innovations:

  1. Wasserstein-2 Kernels: Instead of comparing individual points, the model utilizes kernels operating on probability distributions (mean and variance) within the Wasserstein-2 metric space. This allows for the mathematical propagation of uncertainty through the network.
  2. Affine Operators on Distributions: To maintain computational efficiency, standard convolutions are replaced with distance-preserving affine operators that transform the Gaussian distributions of the hidden layers.
  3. Lipschitz Constraints: By enforcing Lipschitz continuity bounds, we ensure that the "distance" between in-distribution points and outliers does not shrink as data moves deeper into the network, effectively preventing feature collapse.

Key Insights: Uncertainty Decomposition

A core strength of the DistGP approach is its ability to distinguish between different types of uncertainty. Unlike standard models that provide a single confidence score, this framework allows for a nuanced decomposition of why the model might be "unsure."

Visualization of uncertainty decomposition: Epistemic (model knowledge), Aleatoric (data noise), and Distributional (OOD) uncertainty.

As shown in the figure above, the model separates:

  • Epistemic Uncertainty: Uncertainty due to a lack of training data in a specific region (model ignorance).
  • Aleatoric Uncertainty: Uncertainty inherent in the data itself, such as sensor noise or blurry images.
  • Distributional Uncertainty: Uncertainty arising when the input data comes from a different distribution than the training set (e.g., a pathological scan vs. a healthy scan).

Experimental Results

The researchers validated DistGP-Seg using the UK Biobank (UKBB) dataset (nearly 15,000 healthy brain scans) for training and the BraTS 2017 (brain tumors) dataset for OOD evaluation.

  • Segmentation Accuracy: On healthy tissue, DistGP-Seg achieved Dice scores of 0.82–0.86, approaching the performance of standard U-Nets (0.85–0.89) and significantly outperforming previous hierarchical GP methods.
  • OOD Detection Performance: When tested on brain tumors, DistGP-Seg achieved an OOD detection Dice score of 0.512 at a 0.1% False Positive Rate. This vastly outperformed competing methods like DUQ (0.068) and OVNNI (<0.001).
  • Robustness: The model demonstrated a monotonic increase in uncertainty as images were systematically perturbed (e.g., rotated), providing a reliable safety signal that deterministic models lack.

Conclusion and Future Work

The DistGP layer successfully bridges the gap between the high performance of deep convolutional networks and the reliable uncertainty estimation of Gaussian Processes. By maintaining distance awareness through Wasserstein-2 space and Lipschitz constraints, the model provides a robust mechanism for OOD detection in critical medical applications.

Future research will focus on addressing the computational complexity of GP-based models. While sparse approximations (inducing points) make current implementations feasible, matrix-inversion-free inference techniques are needed to scale these layers to higher resolutions and even deeper architectures.

Summary updated

Abstract

Machine learning models deployed on medical imaging tasks must be equipped with out-of-distribution detection capabilities in order to avoid erroneous predictions. It is unsure whether out-of-distribution detection models reliant on deep neural networks are suitable for detecting domain shifts in medical imaging. Gaussian Processes can reliably separate in-distribution data points from out-of-distribution data points via their mathematical construction. Hence, we propose a parameter efficient Bayesian layer for hierarchical convolutional Gaussian Processes that incorporates Gaussian Processes operating in Wasserstein-2 space to reliably propagate uncertainty. This directly replaces convolving Gaussian Processes with a distance-preserving affine operator on distributions. Our experiments on brain tissue-segmentation show that the resulting architecture approaches the performance of well-established deterministic segmentation algorithms (U-Net), which has not been achieved with previous hierarchical Gaussian Processes. Moreover, by applying the same segmentation model to out-of-distribution data (i.e., images with pathology such as brain tumors), we show that our uncertainty estimates result in out-of-distribution detection that outperforms the capabilities of previous Bayesian networks and reconstruction-based approaches that learn normative distributions.
BibTeX
@article{popescu2022distributional,
  author = {Sebastian Popescu and David Sharp and James H. Cole and Konstantinos Kamnitsas and Ben Glocker},
  title = {{Distributional Gaussian Processes Layers for Out-of-Distribution Detection}},
  journal = {The Journal of Machine Learning for Biomedical Imaging},
  year = {2022},
  doi = {10.59275/j.melba.2022-a6bf},
}
Figure