Standard machine learning models for time series, such as Recurrent Neural Networks (RNNs) or standard Transformers, typically assume a "grid" structure: data points are expected at uniform intervals (e.g., every hour). However, real-world data—particularly in Intensive Care Units (ICUs), finance, and scientific sensors—is rarely so polite.
In a clinical setting, a patient’s heart rate might be monitored continuously, but blood glucose is checked every few hours, and specialized lab tests might occur only once every few days. This results in Irregularly Sampled Multivariate Time Series (ISMTS), characterized by extreme sparsity, asynchronous observations across sensors, and varying temporal gaps. Researchers are moving away from simple "data cleaning" (like filling in gaps with the last known value) toward models that treat time as a continuous dimension or learn the underlying "latent" physiology of the subject.
The Problem with the Grid
When data is missing or irregularly spaced, the traditional approach is imputation: filling in the blanks to create a complete matrix. However, as Chester Holtz notes, simple completion techniques can obscure the "informative" nature of the sparsity itself—the fact that a doctor chose to order a test at a specific time is often a signal in its own right. Furthermore, Nan Du and Denny Zhou argue that techniques like last-observation-carried-forward fail to capture the complex, evolving dynamics of a patient's physiological state.
Modeling the Latent Continuous State
One powerful intuition is to stop viewing the time series as a sequence of discrete points and start viewing it as a window into a continuous underlying process.
Marzyeh Ghassemi and David A. Clifton pioneered the use of Multi-Task Gaussian Processes (MTGP) to handle this. By fitting a GP to the sparse data, they can learn model hyperparameters that act as a "latent fingerprint" of a patient’s state. This allows the model to "smooth" over gaps and make predictions based on the learned continuous dynamics rather than the raw, messy observations. Similarly, Margherita Rosnati and Vincent Fortuin combined MTGPs with Temporal Convolutional Networks to predict sepsis, using the GP to provide a consistent input for the neural network regardless of how many data points were missing.
More recently, researchers have turned to Implicit Neural Representations (INRs) and Neural Flows. Batuhan Koyuncu and Isabel Valera introduced TV-INRs, which represent a time series as a continuous function mapped by a neural network. This allows for "amortized" imputation and forecasting in a single pass, effectively treating time as a coordinate rather than an index. To solve the speed issues of traditional Neural ODEs, Mengzhou Gao developed Graph-Structured Neural Flows (GSNF), which capture how different variables (like blood pressure and oxygen levels) interact in a single computational step.
Structural Adaptations for Sparsity
If we want to use modern architectures like Transformers or Graph Neural Networks (GNNs), we must adapt them to be "time-aware."
Arash Hajisafi and Cyrus Shahabi proposed WaveGNN, which uses a decay-aware Transformer to track how the relevance of a sensor reading fades over time, paired with a GNN to model the relationships between different sensors. This ensures that the model doesn't treat a heart rate reading from five minutes ago the same way it treats one from five hours ago.
Another approach is to change the data's "modality" entirely. Zekun Li and Xifeng Yan introduced ViTST, which converts multivariate time series into line graph images. By feeding these images into a Vision Transformer, the model can "see" the patterns and gaps visually. This approach is surprisingly robust; even when half the sensors are missing, the visual patterns of the remaining lines provide enough context for high-accuracy classification.
Probabilistic and Generative Forecasting
Because irregular data is inherently uncertain, many researchers argue that models should output a range of possibilities rather than a single number.
Vijaya Krishna Yalavarthi and Stefan Born developed ProFITi, a model using conditional normalizing flows to learn the joint distribution of future values. This allows the model to capture complex dependencies—for instance, how a drop in one variable might increase the variance (uncertainty) in another. Similarly, Defu Cao and Yan Liu introduced PINFDiT, which uses Diffusion Transformers constrained by physical laws to ensure that even when data is sparse, the model's "hallucinations" about the missing points remain physically plausible.
Why It Works: Capturing the Latent Trajectory
The shared insight across these 31 papers is that the "messiness" of the data is not just noise to be removed, but a sampling of a continuous underlying reality. Whether through Gaussian Processes (Visar Berisha), State-Space Models (Andrew M. Dai), or causal discovery frameworks (Anpeng Wu), the goal is to reconstruct the latent physiological trajectory.
By modeling the patient (or the startup, or the machine) as a continuous entity, these frameworks can:
- Impute naturally: Gaps are filled by the model's understanding of the underlying process.
- Handle asynchrony: It doesn't matter if Sensor A and Sensor B were never recorded at the same time; the model relates them through their shared influence on the latent state.
- Forecast with uncertainty: The model can quantify how much it "knows" based on the density of recent observations.
As Christian Klötergens points out with the Physiome-ODE benchmark, the field is now moving toward more rigorous evaluation, ensuring that these complex neural models actually outperform simple baselines by truly capturing the biological or physical laws governing the data.