Adaptive Conformal Prediction for Non-Stationary Data Streams

This direction is split into other directions: dynamic uncertainty quantification. Its last version stays readable and citable here.

Version 1 of 1207 papers297 conceptsfrozen 2026-10-04

Standard machine learning models are notoriously overconfident. When a model predicts a "90% probability," it often reflects the model's internal state rather than the actual frequency of being correct. Conformal Prediction (CP) was developed to fix this by providing "prediction sets" that are mathematically guaranteed to contain the true answer with a user-specified probability (e.g., 95%).

However, traditional CP relies on a critical assumption: exchangeability. It assumes the future will look like the past. In the real world—where wireless signals fade, robots enter new rooms, and financial markets crash—this assumption fails. This research direction focuses on Online Conformal Prediction, a framework that transforms uncertainty quantification from a static calculation into a dynamic, self-correcting control problem.

The Fragility of Static Calibration

In a typical setup, researchers use a fixed "calibration dataset" to determine a threshold for uncertainty. As shown by Fan Nie, Xiaotian Hou, and James Zou in FactTest, these static thresholds allow Large Language Models (LLMs) to "abstain" from answering when they are likely to hallucinate.

But static thresholds are brittle. If the distribution of user queries shifts, a threshold calculated yesterday might be too risky today. As Osvaldo Simeone and Sangwoo Park argue in their work on wireless communications, AI models in dynamic environments need to recover from temporary under-calibration in real-time.

Uncertainty as a Feedback Loop

The core intuition of this research is to treat the calibration threshold not as a constant, but as a state variable in a feedback loop. If the model makes an error (the true value falls outside the predicted set), the system should become more conservative. If the model is consistently correct, it can afford to be more precise (narrower sets).

This mechanism is often implemented as a form of Online Gradient Descent or Quantile Tracking. For example, in ACon², Sangdon Park and Osbert Bastani use an online feedback loop to adjust price intervals for blockchain oracles. When market volatility spikes, the intervals widen automatically; as the market stabilizes, they shrink.

Similarly, Felipe Areces and John Duchi demonstrate in Online Conformal Prediction via Online Optimization that by explicitly modeling the temporal structure of errors, we can achieve "time-conditional" coverage. This ensures the model isn't just right "on average" over a year, but remains reliable during specific high-risk periods.

Adapting to Non-Stationary Environments

The shift from offline to online calibration enables several key mechanisms for handling non-stationary data:

  1. Sliding Windows: Instead of using all past data, models like EnbPI, developed by Chen Xu and Xu Chen, use a sliding window of recent residuals. This allows the system to "forget" outdated patterns and adapt to current environmental noise.
  2. Temporal Quantile Adjustments (TQA): Zhen Lin and Jimeng Sun proposed TQA, which updates the target quantile at each time step. This is particularly useful for cross-sectional time series, such as monitoring multiple patients in a hospital where individual baselines shift over time.
  3. Error-Driven Regulation: In robotics, Conformal Policy Learning acts as a statistical regulator. It increases caution (widens uncertainty bounds) immediately after a prediction failure, allowing a robot to switch to a "safe mode" when it encounters unfamiliar terrain.

Reliability in the Wild

This adaptive approach is being applied across diverse, high-stakes domains:

  • Wireless Networks: Kfir M. Cohen and Petar Popovski use online CP to schedule low-latency traffic, ensuring that 5G/6G networks meet strict reliability targets even as user demand fluctuates.
  • TinyML: For resource-constrained edge devices, Khalid El Makkaoui introduced TCUQ, which uses a lightweight streaming conformal layer to track data quantiles online without needing to store large historical datasets.
  • Human-AI Collaboration: Shayan Kiyani and George J. Pappas have shown that online calibration can keep AI-human teams safe even when the human expert's reliability or strategy shifts over time.

By treating uncertainty as a dynamic parameter rather than a static property, this research direction provides a path toward AI systems that are not just accurate, but provably reliable in an ever-changing world.

Papers in this version

Showing 60 of 207 papers.

History & citationVersion 1 · 2026-10-04

Split into other directions · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_adaptive_conformal_prediction_for_non_st_01db8556,
  title = {Adaptive Conformal Prediction for Non-Stationary Data Streams},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/adaptive-conformal-prediction-for-non-stationary-data-streams/rdr_01db85561d674f40944f21c8faa8ba49},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_adaptive_conformal_prediction_for_non_st_01db8556_v1,
  title = {Adaptive Conformal Prediction for Non-Stationary Data Streams},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/adaptive-conformal-prediction-for-non-stationary-data-streams/rdr_01db85561d674f40944f21c8faa8ba49/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}