Latent-to-Observable Score Correction for Probabilistic Time Series Imputation

Guojun Liang, Zhan Su, Najmeh Abiri, Slawomir Nowaczyk, Stefan Byttner, Jens Lundström, Prayag TiwariOpenReview 2026 · Submission to ICLR 2026 · Rejected · OpenReview

In the world of multivariate time series—think hospital patient vitals or power grid sensors—missing data isn't just an annoyance; it’s a mathematical trap. While score-based diffusion models have become the gold standard for filling these gaps, a new paper submitted to ICLR 2026 reveals a hidden bias in how we train them.

TL;DR

HSGM (Hierarchical Score-Based Generative Model) fixes a fundamental flaw in diffusion-based imputation: the bias caused by ignoring "naturally" missing data during training. By nesting latent-space diffusion within observation-space refinement, it achieves a new state-of-the-art in accuracy and provides much tighter, more reliable uncertainty estimates.

What Problem Does This Paper Address?

Most modern imputation models (like CSDI) use a "mask-and-predict" strategy. They take a complete dataset, hide some values to simulate missingness, and train the model to recover them. However, real-world datasets already have "original" missing values where the ground truth is unknown.

Commonly, researchers just fill these original gaps with zeros or means during training. The authors of this paper prove—both theoretically and through simulation—that this "ignore it" approach creates a score function bias. Because sensors in a system are often highly correlated, a gap in one sensor distorts the learned "score" (the gradient of the data density) for all other sensors. This bias accumulates during the diffusion process, leading to "blurry" or inaccurate imputations even for the data points we can see.

Methods

To solve this, the authors propose HSGM, a hierarchical framework grounded in the chain rule of probability. Instead of jumping straight to the raw data, HSGM works in two stages:

  1. Latent-Space Diffusion: The model first learns a compressed, "latent" representation of the time series using a Variational Autoencoder (VAE) with Normalizing Flows. This allows it to capture complex, non-Gaussian distributions that standard VAEs miss. A continuous-time diffusion process (VPSDE) then operates in this latent space to reconstruct the "big picture" of the missing data.
  2. Observation-Space Refinement: Once the latent space provides a "clean" guess of the missing values, a second diffusion layer in the observation space refines the details.
  3. Cross-Attention Bridge: To ensure the model doesn't lose the "truth" of the observed points while dreaming up the missing ones, they use a cross-attention mechanism. This balances generative flexibility (creating plausible new data) with reconstruction fidelity (staying true to the sensors that actually worked).

Datasets

The model was put to the test on four diverse benchmarks:

  • P2012 (PhysioNet): A clinical dataset with a massive 80.52% natural missing rate.
  • MIMIC-IV: Large-scale electronic health records with ~49% missingness.
  • ETT (Electricity Transformer Temperature): A high-resolution sensor dataset where the authors simulated "block-missing" patterns (like a sensor failing for an hour).
  • Synthetic Dataset: A custom dataset designed to test how the model handles multiscale trends and seasonality.

Key Insights and Ideas

  • The "Aha" Moment on Bias: The authors rigorously showed that score bias is proportional to the missing rate. In healthcare, where 80% of data might be missing, ignoring the "original" gaps makes it mathematically impossible for a standard diffusion model to learn the true data distribution.
  • Correlation is a Double-Edged Sword: High inter-variable correlation helps us impute data, but it also means that a single "zero-filled" missing value poisons the score estimation for its neighbors.
  • Hierarchical Refinement: By splitting the task into "latent" (global structure) and "observable" (local refinement), the model corrects the bias at the source before fine-tuning the final output.

Key Results

HSGM consistently outperformed all baselines, including popular models like CSDI and LSSDM.

  • Accuracy: On the highly sparse P2012 dataset, HSGM achieved an MAE of 0.241, a significant jump over CSDI (0.301) and the previous best, LSSDM (0.262).
  • Uncertainty: The model achieved the best CRPS (Continuous Ranked Probability Score) scores, meaning its "confidence intervals" were not only narrower but more likely to contain the true hidden value.
  • Visual Proof: Heatmaps of the learned score functions showed that HSGM's gradients almost perfectly matched the ground truth, whereas CSDI's gradients showed significant "drift" as the diffusion steps progressed.

Limitations and Future Work

While HSGM is powerful, it is a multi-stage pipeline (pre-training the NVAE, latent diffusion, and then observation diffusion). This likely makes it more computationally expensive to train than "flat" models.

The authors suggest that future work should focus on further optimizing the latent-space objective. They noted that the likelihood ratio between ground-truth and imputed distributions can be estimated via the latent term, hinting at a path toward even more "principled" imputation strategies that don't require any ground-truth supervision at all.

Summary updated

Abstract

Missing data remains a key challenge in multivariate time series modeling, often degrading downstream performance. Recent score-based generative models show strong potential for high-quality imputations, yet most ignore original missing data during training, since ground truth is unavailable, resulting in biased score estimation. We theoretically analyze the effect of missingness on score-based modeling under the denoising diffusion probabilistic model (DDPM) framework. Our findings reveal that ignoring original missing patterns—especially under high missing rates or strong inter-variable correlations—can significantly distort the learned score function even at non-missing points. To overcome this, we propose the Hierarchical Score-Based Generative Model (HSGM) for probabilistic time series imputation. HSGM integrates latent-space and observation-space diffusion in a layer-wise refinement framework grounded in the chain rule of probability. A pretrained Variational Autoencoder (VAE) with normalizing flows captures complex latent distributions, while a continuous-time variational diffusion (VPSDE) operates in latent space. A cross-attention mechanism between the original and denoised latent states enhances the fidelity and resolution of the generative outputs, while an observation-space diffusion module further refines the final imputations. Experiments on four benchmark datasets show that HSGM achieves the best accurate imputations with tighter uncertainty estimates than existing methods, while effectively correcting score function bias, establishing a new state of the art in time series imputation.
BibTeX
@misc{liang2026latent,
  author = {Guojun Liang and Zhan Su and Najmeh Abiri and Slawomir Nowaczyk and Stefan Byttner and Jens Lundström and Prayag Tiwari},
  title = {{Latent-to-Observable Score Correction for Probabilistic Time Series Imputation}},
  year = {2026},
  url = {https://openreview.net/forum?id=MBMabJjm7M},
  howpublished = {OpenReview},
  note = {Submitted to ICLR 2026},
}
Figure