CalPro: Prior-Aware Evidential-Conformal Prediction with Structure-Aware Guarantees for Protein Structures

Ibne Farabi Shihab, Sanjeda Akter, Anuj SharmaarXiv 2026 · DOI

TL;DR

This paper introduces CalPro, a novel framework that combines evidential learning, conformal prediction, and domain-specific "priors" to provide more reliable uncertainty estimates for protein structure predictions. CalPro significantly improves robustness to distribution shifts and leads to better downstream applications like drug discovery.

What Problem Does This Paper Address?

Deep learning models like AlphaFold have revolutionized protein structure prediction, providing confidence scores (e.g., pLDDT) for their predictions. However, these scores are often not well-calibrated and can degrade significantly when applied to data that differs from the training distribution (e.g., different experimental methods, varying protein lengths, or intrinsically disordered regions). This unreliability is a major bottleneck for high-stakes applications like molecular docking and drug discovery, where accurate uncertainty quantification is crucial for making informed decisions. Existing methods for improving uncertainty estimates, such as Bayesian ensembles or Monte Carlo dropout, are computationally expensive and still struggle with distribution shifts. Conformal prediction offers finite-sample guarantees but typically doesn't leverage rich structural information like protein disorder or flexibility. CalPro aims to bridge this gap by developing a method that is robust to shifts, provides theoretical guarantees, and effectively incorporates domain knowledge.

Methods

CalPro is a hybrid framework with three core components:

  1. Geometric Evidential Head: This component uses a graph neural network (GNN) to process protein structures and outputs parameters for a Normal-Inverse-Gamma (NIG) distribution. The NIG distribution is capable of capturing both aleatoric (inherent randomness) and epistemic (model uncertainty) uncertainty without requiring ensembles. Crucially, this head incorporates structural priors (like disorder and flexibility) directly as node and edge features, rather than as post-hoc adjustments.
  2. Differentiable Conformal Layer: To enable end-to-end training, CalPro uses a differentiable approximation of the conformal quantile step. This allows the model to learn representations that are easier to calibrate. After training, a standard split-conformal procedure is applied to ensure exact finite-sample coverage guarantees.
  3. Prior-Aware Regularizer: Domain priors (e.g., disorder, flexibility) are encoded as soft constraints. This regularizer encourages the model to produce wider uncertainty intervals in regions where priors indicate higher volatility or disorder, ensuring that the uncertainty estimates are not only statistically sound but also aligned with biological intuition.

The overall training objective combines the evidential loss, an evidence regularization term (to prevent overconfidence), the prior-aware regularization, and a soft conformal loss.

Datasets

The paper evaluates CalPro on several datasets:

  • Protein Structure Benchmarks: High-resolution X-ray, NMR, and cryo-EM structures from the Protein Data Bank (PDB) and Electron Microscopy Data Bank (EMDB). These were split to simulate realistic distribution shifts (e.g., temporal separation, avoiding family overlap).
  • FLIP Fitness Tasks: Benchmarks for evaluating mutational fitness landscapes under substantial distribution shift, used to test robustness and downstream decision-making.
  • Non-biological Regression Benchmark: A tabular dataset with synthetic heteroscedastic perturbations to test the generality of CalPro beyond protein structures.

Key Insights and Ideas

  • Priors are Powerful: Incorporating domain knowledge (like protein disorder) directly into the model's uncertainty estimation process, rather than as a post-hoc heuristic, significantly improves robustness and interval sharpness.
  • Evidential + Conformal Synergy: Combining evidential learning (for rich uncertainty representation) with conformal prediction (for finite-sample guarantees) creates a powerful framework that overcomes limitations of each individual approach.
  • Structure-Aware Guarantees: The theoretical framework moves beyond generic Lipschitz bounds for conformal prediction under shift, providing guarantees that are explicitly tied to model complexity and the informativeness of domain priors, making them more relevant for structured data like proteins.
  • Shift Robustness is Achievable: CalPro demonstrates that it's possible to maintain near-nominal coverage and reduce calibration error even under significant distribution shifts, a critical step for real-world deployment.

Key Results

  • Improved Calibration and Shift Robustness: CalPro achieved < 5% coverage degradation across modalities, compared to 15-25% for baselines. It also reduced calibration error by 30-50%.
  • Sharper Intervals: CalPro produced narrower intervals than vanilla conformal prediction in regions where priors were informative, leading to more precise uncertainty estimates.
  • Downstream Performance Gains: CalPro improved ligand-docking success rates by 25% and enabled more sample-efficient active selection of high-value candidates in experimental design.
  • Generalizability: The framework showed promising results on non-biological regression tasks, suggesting its applicability beyond protein structures.

Limitations and Future Work

  • Reliance on Base Predictors: CalPro still depends on the quality of features from base predictors like AlphaFold, inheriting any systematic biases they may have.
  • Conditional Independence: The current conformal layer treats residues (or targets) as conditionally independent, and doesn't model joint coverage constraints across groups of outputs.
  • Priors Can Mislead: While powerful, priors can also be misleading if they are biased or uninformative. Developing diagnostics for "bad priors" is an open problem.
  • Future Work: The authors suggest extending CalPro to structured outputs with joint coverage guarantees, integrating richer classes of priors, and exploring more tightly coupled training objectives. They also see opportunities to apply CalPro in other scientific domains with abundant geometric structure and priors but scarce labels.

Summary updated

BibTeX
@misc{shihab2026calpro,
  author = {Ibne Farabi Shihab and Sanjeda Akter and Anuj Sharma},
  title = {{CalPro: Prior-Aware Evidential-Conformal Prediction with Structure-Aware Guarantees for Protein Structures}},
  year = {2026},
  eprint = {2601.07201},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/2601.07201},
}
Figure