Towards a Certificate of Trust: Task-Aware OOD Detection for Scientific AI
Deep learning is revolutionizing scientific computing, from weather forecasting to fluid dynamics. However, unlike traditional physics-based models that rely on fundamental laws, neural networks are essentially high-dimensional interpolators. When they encounter data far from their training set—known as out-of-distribution (OOD) data—they don't just fail; they often fail confidently, producing "hallucinated" physics that look plausible but are mathematically incorrect.
While OOD detection is well-studied for image classification, it remains a major challenge for regression tasks, which make up the bulk of scientific AI. In regression, the output is a continuous value, making it much harder to quantify "uncertainty" or "trust" in a single prediction. Researchers at ETH Zurich have developed a "certificate of trust" to provide a verifiable metric to tell an end-user: "You can trust this specific prediction," or "This input is too weird; ignore the result."
Methodology: The Joint Likelihood Based Certificate (JLBC)
The authors propose a task-aware OOD detection method. Most existing methods only look at the input ($x$) to see if it looks like the training data. This paper argues that we must also look at the model’s prediction ($y_{pred}$).
- The Joint Distribution: They focus on the joint likelihood $p(x, y_{pred})$. This captures not just whether the input is familiar, but whether the relationship between that input and the resulting prediction is consistent with what the model learned.
- Diffusion Models as Density Estimators: To calculate this likelihood, they use a score-based diffusion model. While these are famous for generating images, they are mathematically equivalent to learning the "score function" (the gradient of the log-density) of a dataset.
- The Probability Flow ODE: By solving a specific differential equation (the Probability Flow ODE) associated with the diffusion model, they can numerically integrate along a path to find the exact log-likelihood of any $(x, y)$ pair. This log-likelihood serves as the "certificate."
- Zero-Shot Detection: The method is "zero-shot" because it doesn't require seeing any OOD samples or ground-truth labels during the detection phase.
Key Insights and Results
- Input-only is insufficient: A key finding is that $p(x)$ alone is a poor certificate. A model might receive a normal-looking input but produce a wildly incorrect output due to task complexity. The joint likelihood $p(x, y_{pred})$ catches these task-specific failures.
- Correlation with Error: The authors found a remarkably strong correlation between the estimated log-likelihood and the actual prediction error (see Figure 1, Panel D). This allows them to fit an exponential curve to the data, providing an a posteriori error estimate—essentially predicting how much error the model made without knowing the right answer.
- High Accuracy: Across scientific datasets including the Wave Equation, Navier-Stokes, and MERRA-2 humidity data, the JLBC achieved an average OOD detection accuracy of ~90%, significantly outperforming baselines.
- Robustness: The method proved robust across different neural network architectures (UNets, ViTs, and FNOs). It does not require changing the underlying regression model; a diffusion model is simply trained alongside it.
Limitations and Future Work
The primary limitation is computational cost. Calculating the likelihood requires solving an ODE, which involves multiple passes through the diffusion denoiser. Additionally, the relationship between likelihood and error is currently heuristic rather than a formal mathematical bound. Future work will focus on optimizing ODE solvers to make certificate generation fast enough for real-time applications like autonomous scientific instruments or live weather monitoring.
Summary updated