Robust and Efficient Conformal Prediction for Reliable Uncertainty

This direction is merged into another direction: continuous uncertainty quantification. Its last version stays readable and citable here.

Version 1 of 117 papers17 conceptsfrozen 2026-10-04

Machine learning models often suffer from a "confidence problem": they are frequently overconfident in their errors, especially when faced with data that differs from their training experience. Conformal Prediction (CP) has emerged as the gold standard for addressing this, providing a framework to turn point predictions into sets or intervals that are guaranteed to contain the true value with a user-defined probability (e.g., 95%).

However, the "standard" version of conformal prediction assumes that the world is static and the data is clean. Recent research is pushing CP into more challenging territories—handling adversarial attacks, distribution shifts, and data scarcity—while striving to make these guarantees computationally efficient.

The Core Intuition of Conformal Prediction

At its heart, conformal prediction works by calculating a nonconformity score, which measures how "unusual" a new data point looks relative to a calibration set. If a model is 95% confident, CP finds a threshold such that 95% of the calibration data has a lower nonconformity score. For a new test point, CP includes every possible label that would result in a score below that threshold.

The power of this approach is that it is "distribution-free"—it doesn't care if your model is a neural network or a decision tree, as long as the data is exchangeable. But as researchers like Ulf Johansson and Henrik Boström have shown in Interpretable and specialized conformal predictors, we can go further by "coaching" simple models with complex ones to create prediction sets that are both mathematically trustworthy and highly efficient.

Guaranteeing Safety Under Attack

Standard CP assumes the data hasn't been tampered with. In high-stakes environments, this is a dangerous assumption. Two primary threats exist: data poisoning (where an adversary corrupts the training/calibration set) and adversarial examples (where an adversary tweaks a specific test input).

To combat poisoning, Yan Scholten and Stephan Günnemann introduced Reliable Prediction Sets (RPS) in Provably Reliable Conformal Prediction Sets in the Presence of Data Poisoning. Their intuition is to use data partitioning and majority-vote aggregation. By splitting the data, they ensure that a limited number of poisoned samples cannot exert enough influence to break the statistical validity of the entire system.

For test-time attacks, Thomas Massena and colleagues developed lip-rcp in Efficient Robust Conformal Prediction via Lipschitz-Bounded Networks. By using neural networks with a bounded Lipschitz constant—essentially a mathematical "speed limit" on how much the output can change for a given change in input—they can provide prediction sets that remain valid even if the input is maliciously perturbed.

Adapting to a Shifting World

One of the strictest requirements of CP is "exchangeability"—the idea that the future will look like the past. In fields like drug discovery, this is rarely true; new molecules often look nothing like the ones used for training.

In CoDrug: Conformal Drug Property Prediction with Density Estimation under Covariate Shift, Siddhartha Laghuvarapu and Jimeng Sun tackle this "covariate shift." They use Energy-Based Models (EBMs) and Kernel Density Estimation to weight the importance of calibration points. If a calibration point is similar to the new test point, it gets more "vote" in determining the threshold. This allows the model to maintain its coverage guarantees even when the data distribution drifts.

Efficiency and the Regression Frontier

While these advancements have solidified CP for classification, a recurring theme across the literature—from Hao Zeng to Alireza Javanmardi—is the regression gap.

Most current robust CP methods are designed for discrete labels (e.g., "Is this a cat or a dog?"). Extending these to continuous values (e.g., "What is the binding affinity of this drug?") is difficult because:

  1. Infinite Candidate Sets: In classification, you can test every label. In regression, you can't test every real number.
  2. Loss Formulations: As Sangwoo Park and Osvaldo Simeone note in Optimizing In-Context Learning for Efficient Full Conformal Prediction, making CP "differentiable" so it can be optimized by a Transformer requires new loss functions that can handle intervals rather than discrete sets.

The current frontier of the field is focused on bridging this gap, ensuring that whether a model is predicting a category or a continuous physical property, it can do so with a mathematically backed "I don't know" when the data becomes unreliable.

Papers in this version

History & citationVersion 1 · 2026-10-04

Merged into another direction · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_robust_and_efficient_conformal_predictio_ddda506d,
  title = {Robust and Efficient Conformal Prediction for Reliable Uncertainty},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/robust-and-efficient-conformal-prediction-for-reliable-uncertainty/rdr_ddda506d5ec941299060a005bee1373a},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_robust_and_efficient_conformal_predictio_ddda506d_v1,
  title = {Robust and Efficient Conformal Prediction for Reliable Uncertainty},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/robust-and-efficient-conformal-prediction-for-reliable-uncertainty/rdr_ddda506d5ec941299060a005bee1373a/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}