Censoring-Aware Online Conformal Prediction for Event Streams

Many systems need to forecast when the next event will happen: the next hospital admission, the next post from a user, the next aftershock. A model that does this is called a temporal point process, a model of events scattered in continuous time. For such forecasts to be usable, each one should come with an interval that contains the true arrival time a promised fraction of the time, say 90%. Conformal prediction provides that promise. It is a wrapper that turns any model's errors on held-out data into intervals with guaranteed coverage. The only existing conformal method for neural point processes (Distribution-Free Conformal Joint Prediction Regions for Neural Marked Temporal Point Processes) assumes the past and future are statistically interchangeable, and its authors acknowledge that real event streams break this assumption. Time-series methods that drop the assumption, such as adaptive conformal inference, adjust the interval width online after each observed hit or miss. They were built for regularly spaced series, though, and in an event stream the timing of that feedback is not innocent.

The observation behind this proposal is simple. Suppose an interval [a, b] is issued for the next arrival. If the event lands early, we learn the outcome immediately. If the event is late, and especially if it is later than b (a miss), we learn the outcome only when it finally arrives, which may be much later. When many sequences share one calibrator, as with thousands of users or patients, the feedback that reaches it in calendar time is biased toward short gaps. That bias is worst during a slowdown, when the misses are exactly the late-reporting ones. A calibrator that updates only when events arrive therefore reacts slowest to the shift it most needs to catch. We propose to update the calibrator at the moment the outcome is decided, which for an upper-bound miss is the moment the clock passes b with no event. Doing this cheaply for many concurrent sequences requires a point process whose cumulative intensity can be evaluated in closed form, and that is where tractable neural point processes come in.

Core Claim

Online conformal calibration for asynchronous event streams should run in calendar time and treat each open interval as censored feedback. An upper-bound miss counts the instant its deadline passes. A hit counts at arrival. The rate at which the target level adapts is set per unit of elapsed time rather than per event. Combined with a non-conformity score computed from a closed-form cumulative hazard, this should match event-indexed adaptive conformal inference when event rates rise and clearly outperform it when rates fall. On slowdowns it should give fewer and shorter runs of under-coverage at similar interval width.

The score comes from the time-rescaling idea. Let $\Phi(t)$ be the model's cumulative hazard, meaning the integrated intensity since the last event $t_{i-1}$, given the history. The quantity

$$u_i = 1 - \exp\big(-[\Phi(t_i) - \Phi(t_{i-1})]\big)$$

is the model's predicted probability that the next event has already happened by $t_i$. If the model is correct, $u_i$ is uniform on [0, 1]. Under drift it is not, and its departures are serially correlated. Intervals are formed as quantile ranges in $u$-space and mapped back to time by inverting the monotone $\Phi$. Whether the upper bound has been crossed is then a direct comparison of the clock against a stored time. No integral is evaluated in the monitoring loop.

Research Program

The model side reuses the monotone cumulative-hazard network of Fully Neural Network based Model for General Temporal Point Processes. That network models $\Phi$ directly, with positive weights, conditioned on a recurrent history encoding, so likelihood and $u_i$ each cost one forward pass. For marked data we use the integral estimation module of Addressing Mark Imbalance in Integration-free Marked Temporal Point Processes, which models the joint mark-time cumulative function in the same integration-free way. Both papers supply the method. Neither produces intervals.

Mapping a $u$-quantile back to a time requires root-finding. Because $\Phi$ is monotone, bisection is guaranteed to converge and costs a few dozen evaluations of a small output head per bound, not of the whole history encoder. We measure this cost against the quadrature that a Transformer Hawkes Process needs and against the Langevin sampling of SMASH, which takes hundreds of steps per prediction. We do not claim to remove inversion entirely. That inversion is the latency cost Dheur et al. flag, and our contribution is to confine it to the moment an interval is issued and to make it cheap there.

The calibration side generalizes Adaptive Conformal Inference Under Distribution Shift. That method lowers or raises a working miscoverage level $\alpha_t$ after each miss or hit. We change three things. First, feedback events are "resolution times": the arrival time for hits and lower-bound misses, and the deadline $b$ for upper-bound misses. Second, the step size is scaled by calendar time elapsed since the last update, so adaptation speed no longer depends on how bursty the stream is. Third, following Adaptive Conformal Predictions for Time Series, we run several time-scaled step sizes in parallel and combine them with Bernstein online aggregation, which removes hand-tuning of the step size.

The analysis has a modest goal. ACI's long-run coverage bound is deterministic and holds for any sequence of feedback, so it carries over to resolution-time ordering. Delays are bounded by interval length once $\alpha_t$ is clipped away from zero, which also fixes ACI's infinite-interval failure. The new statement we will try to prove is that event-indexed feedback with arrival-time ordering produces a transient coverage deficit after a rate drop that grows with the slowdown factor, and that censoring-aware feedback removes it. Mark sets follow the joint highest-density construction of Dheur et al. in a second phase. The time interval is the primary target.

Baselines

The naive reference is the point process's own predictive quantiles, used as intervals without any calibration. Dheur et al. report that such raw-output heuristics "consistently failed to provide these reliability guarantees." We run it on the identical trained model, so any gap reflects calibration alone.

The strongest existing method for this data type is static split conformal from Distribution-Free Conformal Joint Prediction Regions for Neural Marked Temporal Point Processes, using its quantile-regression and highest-density variants. It reports meeting nominal 90% coverage on LastFM, Reddit, Github, Wikipedia, MOOC and MIMIC2. We found no reported interval-width figure on Lacuna for C-HDR or C-QR, so we reproduce both under their preprocessing (50 most frequent marks) with the same base model, the same calibration split, and the same model-tuning budget. Beating it means equal or better long-run coverage, better worst-window coverage after shifts, and no more than comparable width.

The strongest adaptive competitor is event-indexed AgACI on the same $u_i$ scores, ordered by arrival time across pooled sequences. This is our proposal with the censoring-aware, time-scaled update removed, so it is the ablation that isolates the key ingredient. It receives the same expert grid and aggregator. A second ablation keeps censoring-aware feedback but uses per-event steps, which separates the contribution of early miss detection from that of time scaling.

We also include Sequential Predictive Conformal Inference applied to the sequence of $u_i$ values. It reports intervals 30–50% narrower than EnbPI at maintained 90% coverage on regular series. It represents the "model the score autocorrelation" route, and it gets the same lag window and refit schedule.

Why not just…?

Why not just run ACI or AgACI over events in arrival order? This is the real threat, and it is built in as the central ablation. When rates rise, or are stationary, we expect it to perform about as well as our method, because feedback delays are then short. Our claim fails if it also matches us on rate drops. The synthetic slowdown experiments below are designed to settle that, and we will report the outcome either way.

Why not just recalibrate the model, for example by refitting an isotonic map from $u$ to empirical frequency on a sliding window, or by fine-tuning online? Both still learn from arrived events only, so they inherit the same survivorship bias during slowdowns. Neither carries a coverage guarantee under arbitrary drift. We include sliding-window isotonic recalibration as an extra baseline to check whether the bias matters in practice, not just in principle.

Why not just use SPCI-style score forecasting? SPCI exploits autocorrelation in scores, but it orders them by index and observes them only at arrival. It also refits a quantile forest at every step, which is the cost its own literature flags. If SPCI on $u_i$ achieves narrower intervals at matched coverage, that suggests the two ideas compose. We then test SPCI's quantile forecast fed by censoring-aware feedback rather than treating SPCI as a rival.

Proposed Experiments

The decisive check is synthetic. We generate many concurrent Hawkes1/Hawkes2-style sequences, the multivariate Hawkes generators with known kernels used by Omi et al. and Dheur et al., over a shared calendar. We inject controlled regime changes: a drop in baseline rate, a drop in branching ratio, the mirror-image increases, and a mark-mix shift. The slowdown factor is varied from 2 to 20. The prediction is an asymmetric result. Our method and event-indexed AgACI should tie on increases, and on decreases ours should cut post-shift under-coverage, measured as the integrated deficit below 90% and the longest miss run, by an amount that grows with the slowdown factor. A symmetric result would falsify the core claim.

On real data we use StackOverflow, LastFM, Reddit, MOOC, Wikipedia and MIMIC2 as in Dheur et al., plus Retweets. Here Transformer Hawkes Process reports a log-likelihood of −2.04 against −5.60 for the Neural Hawkes Process, which serves as a sanity check that our base model is competitive. A known risk is that conformal studies often standardize timestamps per sequence. We need raw calendar timestamps so that sequences genuinely overlap, and we will report any dataset where they are unavailable.

Metrics are long-run coverage, coverage over sliding calendar windows (worst window and fraction of windows below 85%), median interval length in log-time, time from a shift to recovered coverage, and wall-clock cost per issued interval. For cost, the monotone-hazard model with bisection is compared against quadrature-based likelihood models, with SMASH's Langevin sampling as the slow reference. A secondary experiment extends the method to joint time–mark regions and checks that the sharpness advantage of C-HDR over Bonferroni-combined independent regions survives online adaptation.