Towards a Certificate of Trust: Task-Aware OOD Detection for Scientific AI

Bogdan Raonic, Siddhartha Mishra, Samuel Lanthaler

Introduction

Deep learning is revolutionizing scientific fields like fluid dynamics and weather forecasting, often outperforming traditional numerical solvers in speed and efficiency. However, these data-driven models suffer from a critical "closed-world" problem: they act as interpolative engines. When they encounter data significantly different from their training set—known as out-of-distribution (OOD) samples—they do not simply fail; they fail silently, providing confident but wildly inaccurate predictions.

While OOD detection is a well-studied problem for image classification (e.g., distinguishing a cat from a toaster), it remains severely underexplored for regression tasks, which constitute the vast majority of scientific AI applications. In scientific computing, researchers do not just need a prediction; they need to know if they can trust it. This paper introduces a verifiable "certificate of trust" to ascertain model reliability without requiring ground-truth answers, filling a crucial gap in the deployment of AI for scientific discovery.

Methodology: The Joint Likelihood Based Certificate (JLBC)

The core innovation of this work is the Joint Likelihood Based Certificate (JLBC). Traditional OOD methods often look only at the input data ($x$) to see if it appears anomalous. The authors argue that for regression, one must evaluate the input and the model's prediction ($y_{pred}$) together.

Overview of the Joint Likelihood Based Certificate pipeline, showing diffusion model training, certificate computation, and the strong correlation between the certificate and prediction error across various scientific datasets.

The JLBC framework operates through the following steps:

  1. The Diffusion Model: A score-based diffusion model (specifically a denoiser) is trained on the joint distribution $p(x, y)$ of the training data. Crucially, this training is entirely independent of the scientific regression model being evaluated.
  2. Probability Flow ODE: At test time, a new input $x$ is passed through the scientific AI model to get a prediction $y_{pred}$. The diffusion model’s "probability flow ODE" is then used to numerically estimate the joint log-likelihood $\log p(x, y_{pred})$.
  3. Task-Awareness: Because it evaluates the pair $(x, y_{pred})$, the certificate is inherently "task-aware." If a model produces a physically nonsensical output for a normal-looking input, the joint likelihood will plummet, flagging a potential failure.
  4. Zero-Shot Detection: This process is "zero-shot" for OOD detection—it requires no access to ground-truth labels for the test data to determine if a prediction is trustworthy.

Experimental Validation

The researchers validated their approach across a diverse range of complex scientific and benchmark datasets:

  • PDE Datasets: 2D Wave equations and Navier-Stokes equations (fluid flow) with various initial conditions (Sines, Gauss, Shear Layer, etc.).
  • MERRA-2 Humidity Forecast: Real-world satellite data used to forecast surface-level specific humidity.
  • Medical Imaging: Brain tumor segmentation using the BraTS2020 dataset (3D MRI volumes).
  • Computer Vision Benchmarks: CIFAR10 and MNIST to demonstrate extensibility to classification.

Key Insights and Discoveries

A major "aha" moment in the paper is demonstrating that simply checking if the input $p(x)$ is in-distribution is insufficient. In scientific regression, a model might receive a perfectly normal input but produce a high-error output because the specific task is difficult or the model hasn't learned that specific regime.

Scatter plot showing that relying solely on input likelihood (log p(x)) fails to isolate high-error predictions, as some high-error samples still exhibit high input likelihood.

By capturing the "intrinsic complexity" of the task, the joint likelihood $p(x, y)$ provides a much more reliable metric. The researchers found a remarkably strong correlation between the estimated joint likelihood and the actual $L_1$ prediction error. This allows the likelihood to serve as a direct proxy for accuracy.

Furthermore, the JLBC is architecture-agnostic. It works regardless of whether the underlying scientific model is a U-Net, a Vision Transformer (ViT), or a Fourier Neural Operator (FNO). If a user has access to a tiny sliver of ground-truth data (as few as 64 samples), they can even fit an exponential curve to the likelihood-error relationship to provide a quantitative a posteriori estimate of the expected error for all future predictions.

Performance and Results

The JLBC demonstrated exceptional performance across multiple domains:

  • High Detection Accuracy: On the Navier-Stokes "NS-PwC" dataset, the JLBC achieved an AUROC of 0.999 and a classification accuracy of 99.4% for OOD detection.
  • Superiority over Bayesian Methods: The approach significantly outperformed standard uncertainty estimation methods like MC-Dropout and "Rate-In" in both accuracy and speed. For the Wave Equation, JLBC was roughly 5x faster than the Rate-In method.
  • Robustness in Medicine: In brain tumor segmentation, the method successfully identified OOD samples caused by different MRI modalities (e.g., switching from FLAIR to T2 scans) that would otherwise lead to massive, silent segmentation errors.
  • Consistent Reliability: Across all scientific datasets tested, the method maintained an average accuracy of ~90% and a very low False Positive Rate (~3.3%).

Limitations and Future Directions

While highly effective, the authors acknowledge that the relationship between joint likelihood and error is currently heuristic and does not constitute a formal mathematical proof of error bounds. Additionally, while relative likelihood values are excellent for flagging OOD samples within a single model, absolute values cannot currently be compared across different models to determine which is objectively better.

Finally, although inference is fast, the method requires training a separate, high-quality diffusion model on the joint distribution, adding to the initial computational overhead. Future work will explore more formal calibration techniques, such as conformal prediction, to translate these likelihood scores into rigorous statistical guarantees.

Summary updated

Abstract

Data-driven models are increasingly adopted in critical scientific fields like weather forecasting and fluid dynamics. These methods can fail on out-of-distribution (OOD) data, but detecting such failures in regression tasks is an open challenge. We propose a new OOD detection method based on estimating joint likelihoods using a score-based diffusion model. This approach considers not just the input but also the regression model's prediction, providing a task-aware reliability score. Across numerous scientific datasets, including PDE datasets, satellite imagery and brain tumor segmentation, we show that this likelihood strongly correlates with prediction error. Our work provides a foundational step towards building a verifiable 'certificate of trust', thereby offering a practical tool for assessing the trustworthiness of AI-based scientific predictions.
BibTeX
@inproceedings{raonic2026towards,
  author = {Bogdan Raonic and Siddhartha Mishra and Samuel Lanthaler},
  title = {{Towards a Certificate of Trust: Task-Aware OOD Detection for Scientific AI}},
  booktitle = {International Conference on Learning Representations},
  year = {2026},
  url = {https://openreview.net/forum?id=2RuSWLQK82},
}
Figure