Probabilistic Imputation of Irregularly Sampled Time Series

39 concepts31 papers1 proposal

Deep learning architectures like Recurrent Neural Networks (RNNs) and Temporal Convolutional Networks (TCNs) are exceptionally powerful at sequence modeling, but they share a rigid assumption: they expect data to arrive on a uniform grid. They assume that the time step between index $t$ and index $t+1$ is always identical, and that every variable is observed at every step.

Real-world data rarely complies. In an Intensive Care Unit (ICU), a patient's heart rate might be monitored continuously, while critical blood tests are ordered hours or days apart at irregular intervals. In astronomy, a telescope can only record the brightness of a supernova when the sky is clear and the target is visible, leading to highly fragmented, asynchronous observations across different color filters.

When faced with such "messy" data, traditional approaches rely on deterministic imputation, such as forward-filling (carrying the last observed value forward) or linear interpolation. However, these heuristics treat imputed values as absolute ground truth. They throw away two vital pieces of information: how much time has elapsed since the last true observation, and how uncertain we are about the current state.

To bridge this gap, a powerful paradigm has emerged: using Gaussian Processes (GPs) as an uncertainty-aware, continuous preprocessing layer that transforms irregular, noisy observations into a regularized grid suitable for deep learning.

From Point Estimates to Distributions over Functions

Instead of guessing a single point value for a missing timestamp, we can model the entire time series as a continuous probability distribution. This is the core intuition behind using Gaussian Process Regression (GPR) for time-series interpolation, as established by Benjamin M. Marlin et al..

A Gaussian Process defines a distribution over functions, characterized by a mean function and a kernel (covariance) function. The kernel dictates how points correlate with one another over time. When we condition a GP on a set of sparse, irregularly sampled observations, it yields a posterior distribution. For any arbitrary timestamp—whether an observation exists there or not—the GP posterior provides two outputs:

  1. A mean estimate ($\mu$), representing the most likely value of the signal.
  2. A variance ($\sigma^2$), representing a mathematically rigorous measure of uncertainty.
Deterministic (Forward-Fill):  [Value] ------> [Value] ------> [Value] (No uncertainty)
Probabilistic (GP):            [Value] ~~~(High Variance)~~~> [Value] (Uncertainty grows with time)

As time passes without a new observation, the variance naturally expands, signaling that the model is less certain about the state. When a new data point is observed, the variance shrinks back down.

This global functional fit captures non-linear trends and preserves underlying signals far better than local smoothing heuristics, a property demonstrated by Mohamed A. Naiel et al. in preserving causal signals and Hayley Hung et al. in tracking sparse behavioral poses. In astronomy, this allows researchers to reconstruct continuous "light curves" from sparse telescope observations. For instance, K. Abe et al. and Y. Sharma et al. utilize GPR to precisely determine the explosion onset and peak magnitudes of supernovae despite highly irregular observation cadences. Similarly, Qi Li 0006 et al. leverage this probabilistic interpolation to capture underlying behavioral trends from sporadic social media activity.

Sharing Strength Across Channels via Multitask GPs

Real-world time series are rarely isolated; they are multivariate. In clinical settings, physiological signals like heart rate, blood pressure, and respiratory rate are deeply interconnected. In astrophysics, a transient cosmic event is observed simultaneously across multiple spectral bands.

If we fit independent GPs to each channel, we miss a massive opportunity: channels correlate with one another. A Multitask Gaussian Process (MGP)—or Multi-Output GP—solves this by utilizing a joint covariance function that models both temporal correlations and cross-channel correlations.

As Joseph D. Futoma et al. demonstrate, this architecture enables cross-imputation. If a critical lab value is missing at a specific hour, the MGP does not just look at past values of that same lab test; it infers the missing value by looking at other, more frequently measured physiological variables that correlate with it. This information sharing significantly reduces uncertainty in unobserved regions.

This multi-output capability allows for continuous state estimation and trajectory forecasting across highly asynchronous streams. Li-Fang Cheng et al. leverage this to quantify patient trajectory uncertainty and optimize ICU lab-ordering policies, while Wancen Mu et al. extend the framework to explicitly capture "lead-lag" effects, where one biological signal systematically follows another after a temporal delay.

Feeding Uncertainty to Downstream Neural Networks

How do we connect this continuous, probabilistic GP representation to standard deep learning classifiers?

The standard design pattern is to use the MGP as an uncertainty-aware front-end adapter. The pipeline operates in three distinct phases:

  1. Probabilistic Fit: The raw, irregularly sampled, multi-channel data is passed to the MGP layer.
  2. Regularized Resampling: The MGP posterior is queried at a set of regularly spaced intervals (e.g., every 30 minutes), creating a uniform temporal grid.
  3. Dual-Input Processing: For every variable at every grid point, the MGP outputs both the mean and the variance. This regularized matrix of means and variances is then fed directly into a downstream neural network.
Irregular Data ──> [ Multitask GP ] ──> Uniform Grid (Means + Variances) ──> [ RNN / TCN ] ──> Prediction

Why is passing the variance so critical? Work by Michael Moor et al. and Margherita Rosnati et al. shows that the confidence level of an imputed value is a vital feature for downstream classification. When the neural network encounters a grid cell with high variance, it learns to down-weight that input, treating it as uninformative noise and relying instead on historical context or correlated channels. Conversely, when the variance is low, the network can confidently react to sudden changes in the signal.

This end-to-end integration has driven major breakthroughs in early diagnostic systems. By combining MGPs with recurrent or convolutional networks, Joseph Futoma et al. and Michael Moor et al. developed early-warning systems capable of predicting life-threatening sepsis hours before clinical onset, drastically outperforming models that rely on standard imputation. Similarly, Changchang Yin et al. utilize time-aware recurrent architectures to perform probabilistic imputation, treating missing data as a distribution to map out patient subphenotypes during the highly uncertain "cold-start" phase of hospital admission.

The exact same mathematical mechanism translates seamlessly to astrophysics. To handle the massive data deluge from wide-field surveys, Miles Cranmer et al., Renée Hložek et al., and Ho-Sang Chan et al. employ multivariate GP interpolation to normalize asynchronous multi-band light curves into uniform grids. This allows deep learning models to extract standardized features and flag rare cosmic explosions or anomalies up to a week before they reach peak brightness.

By shifting from deterministic point-imputation to probabilistic GP interpolation, these frameworks ensure that downstream models are never blinded by missing data, but are instead fully aware of what they know—and what they do not.

Key Foundational Work

Learning from time series: Supervised Aggregative Feature Extraction (2012) — contributed 2 concept elements to this direction.
Hierarchical Bayesian modelling of gene expression time series across irregularly sampled replicates and clusters (2013) — contributed 2 concept elements to this direction.
A subsystems approach for parameter estimation of ODE models of hybrid systems (2012) — contributed 1 concept elements to this direction.
History & citationVersion 1 · 2026-10-04

Current · Permanent link

View frozen version 1

Cite this direction

Latest version (a living page):

@misc{lacuna_multitask_gaussian_process_imputation_56f146f0,
  title = {multitask Gaussian process imputation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/multitask-gaussian-process-imputation/rdr_56f146f0393f4511ba5089e99eb0f804},
  note = {Latest version of a living research direction; cite a version for a fixed text. Accessed 2026-11-20},
}

This exact version (version 1, frozen text):

@misc{lacuna_multitask_gaussian_process_imputation_56f146f0_v1,
  title = {multitask Gaussian process imputation},
  author = {{Lacuna}},
  howpublished = {Lacuna research direction},
  year = {2026},
  url = {https://lacuna.tiptreesystems.com/direction/multitask-gaussian-process-imputation/rdr_56f146f0393f4511ba5089e99eb0f804/v/1},
  note = {Version 1, frozen 2026-10-04. Accessed 2026-11-20},
}
About

These directions overlap with this one through shared papers.

See something wrong? Report a mistake