TL;DR
Generating synthetic time series from irregular, real-world data usually relies on "one-shot" imputation, which introduces structural biases that cripple generative quality. This paper introduces a co-evolving Monte Carlo EM framework that iteratively refines data completions alongside a diffusion-based generator, "closing the loop" to achieve state-of-the-art performance across diverse corruption regimes.
What Problem Does This Paper Address?
In fields like healthcare and finance, time-series data is rarely perfect. Sensors fail, sampling rates vary, and observations often arrive at uneven intervals. To train generative models on this "irregular" data, researchers typically use an "open-loop" strategy: they use a pre-trained imputer to fill in the gaps once, freeze those completions, and then train a generator on the resulting "complete" dataset.
The authors argue this strategy is fundamentally broken. Traditional imputers often produce "point estimates" (like a conditional mean), which collapse the natural ambiguity of missing data into a single, overly smooth average. Even stochastic imputers become "stale" because they don't adapt as the generator improves. This leads to a mismatched training set that narrows the learned distribution, resulting in poor-quality synthetic data.
Methods
The core contribution is a co-evolving Monte Carlo EM (MCEM) framework. Instead of a one-off preprocessing step, the imputer and generator evolve together through two alternating steps:
- E-step: Missing values are sampled from the current generator’s posterior distribution. This ensures the completions are always consistent with the model's latest understanding of the data.
- M-step: The generator is retrained on these fresh completions using standard unconditional training.
To leverage powerful vision-based diffusion models, the authors "lift" 1D time series into 2D image-like representations (e.g., using delay embeddings). However, this creates a "dual-space" problem: the generator's prior lives in 2D image space, but the observations live in 1D time-series space.
To bridge this gap, they introduce Posterior Sampling for Lifted Representations (PSLR). PSLR enforces three critical structural conditions during the E-step:
- C1: Operator-consistent conditioning: Ensuring updates are calculated using the true mapping between 1D and 2D spaces.
- C2: Uncertainty preservation: Maintaining the full diversity of possible completions rather than collapsing to a point estimate.
- C3: Manifold consistency: Ensuring the generated 2D images actually correspond to valid, consistent 1D time series.
Datasets
The authors evaluated their method on a standard 10-dataset benchmark including ETTh1/2, ETTm1/2, Weather, Electricity, Energy, Sine, Stock, and Mujoco.
Crucially, they tested across four distinct corruption regimes to simulate real-world messiness:
- Timestamp-level masking: Entire time steps are dropped.
- Per-cell asynchronous masking: Individual features drop independently.
- Contiguous block gaps: Long segments of data are missing (e.g., a sensor blackout).
- Continuous-time resampling: Data is sampled on an irregular, fractional grid.
Key Insights and Ideas
- Iteration is the Binding Constraint: The authors prove that the primary driver of generative fidelity isn't the specific architecture or imputer design, but the iteration between the two. Closing the loop removes irreducible biases that "open-loop" pipelines cannot escape.
- The Imputation-Generation Paradox: Better imputation error (like lower MSE) does not lead to better generation. In fact, point imputers with very low MSE often produce the worst generators because they destroy the data's natural variance.
- Dual-Space Geometry Matters: When working with "lifted" representations, naive posterior samplers fail because they ignore the cross-space constraints. PSLR provides the necessary mathematical "glue" to make image-space diffusion work for time-series observations.
Key Results
The proposed method achieved a new state-of-the-art (SOTA) across all benchmarks:
- 59% relative improvement in discriminative score over the strongest baselines.
- 73% improvement in Context-FID, a metric for representation-level fidelity.
- 71% reduction in feature-correlation error.
- Rapid Convergence: The model typically reaches optimal performance in just ~5 EM iterations, making it significantly faster than some complex open-loop baselines like KoVAE.
Limitations and Future Work
- Identifiability: In cases of extreme data loss (e.g., 70% block-start missingness), the observed data may not uniquely determine the true distribution. The model can match the observations perfectly but still differ from the "ground truth" in the missing regions.
- Local Optima: Like all EM-based methods, the final solution can depend on the initial starting point.
- Computational Cost: While faster than some, the framework is roughly 2.2x slower than the fastest single-pass diffusion models due to the iterative nature of the EM loop.
- Future Directions: The authors suggest amortizing the posterior sampling to speed up training and extending PSLR to handle non-linear lifting transformations.
Summary updated