In fields like clinical medicine or environmental monitoring, data doesn't arrive on a neat, regular schedule. A patient's heart rate might be monitored continuously, while lab results arrive hours or days apart. Handling this "irregularity" usually requires complex, memory-hungry models.
TRESS (Timestamped Recent Events and Statistical Summaries) offers a refreshing alternative: a compact, fixed-size representation that achieves state-of-the-art accuracy with a tiny fraction of the parameters used by traditional models.
TL;DR
TRESS is a highly efficient forecasting model for irregular time series that represents history using a fixed number of recent events combined with global statistical summaries. By conditioning its predictions on the specific query time, it achieves the lowest error rates on four major benchmarks while being up to 2.6x faster during inference than existing methods.
What Problem Does This Paper Address?
Most time series models assume data is sampled at regular intervals. When gaps are irregular, models must account for both what was measured and when.
Existing approaches often rely on continuous-time latent states (like Neural ODEs) or complex attention mechanisms. These can be computationally expensive, especially in "multi-query" scenarios—for instance, when a doctor wants to see a patient's predicted status at 1 hour, 4 hours, and 12 hours into the future. Re-processing the entire patient history for every single query is a massive waste of compute.
The authors identify a key challenge: how to create a history representation that stays the same size as the record grows, remains accurate across different forecast horizons, and allows for efficient repeated queries.
Methods
TRESS consists of two primary modules designed to handle the "where" and "when" of irregular data:
- Query-conditioned Event Lifting (QEL): Instead of just looking at raw values, QEL centers the $K$ most recent observations (the "events") around the latest known value. It then uses an "event gate" to weight these deviations based on two factors: the age of the observation and the forecast horizon. A deviation from an hour ago might be critical for a 10-minute forecast but irrelevant for a 10-hour one.
- Channel-conditioned Joint Decoding (CJD): This module combines the local features of a specific variable (channel) with a shared "context" derived from all other variables. Crucially, it uses an age-count gate to decide how much to trust this cross-variable context. If a variable hasn't been measured in a long time (it's "stale"), the model leans more heavily on the context provided by other, more recently updated variables.
To ensure no information is lost from the distant past, TRESS also maintains a set of Statistical Summaries (mean, variance, min, etc.) for the entire history of each variable. This allows the model to stay "fixed-size" regardless of whether the history spans minutes or years.
Datasets
The authors evaluated TRESS on four diverse benchmarks for Irregularly Sampled Multivariate Time Series (IMTS):
- HumanActivity: Sensor data for activity recognition.
- USHCN: Daily weather measurements from the US Historical Climatology Network.
- PhysioNet: Clinical ICU data from the 2012 PhysioNet Challenge.
- MIMIC-III: A large-scale database of de-identified clinical data from ICU stays.
Key Insights and Ideas
- Persistence as a Prior: TRESS predicts a correction to the latest observed value rather than the value itself. This "unit prior" means the model starts with a sensible baseline (the last known state) and only learns the necessary adjustments.
- The Power of Stale Data: One of the most interesting findings is that cross-variable context is most valuable when the target variable's own data is old. As shown in Figure 4, the "Unseen" and "Older" groups see the biggest accuracy gains from the shared context branch.
- Efficiency through Reuse: Because the history representation is decoupled from the query time, the "history projection" can be computed once and cached. Subsequent queries for different future times only need to run the lightweight decoder, leading to significant speedups.
Key Results
- State-of-the-Art Accuracy: TRESS achieved the lowest Mean Squared Error (MSE) and Mean Absolute Error (MAE) across all four datasets, outperforming much larger models like Warpformer and APN.
- Tiny Footprint: TRESS uses between 1,574 and 9,457 parameters—orders of magnitude fewer than many competitors.
- Inference Speedup: For a workload of 32 successive queries, TRESS reduced CPU inference time by a factor of 2.12–2.60x compared to running the full model for each query.
- Memory Efficiency: It requires significantly less extra GPU memory (as little as 1.2 MiB), making it ideal for deployment on resource-constrained hardware.
Limitations and Future Work
- Fixed Event Budget: While the statistical summaries help, the model's "resolution" for recent history is limited by the budget $K$ (typically 8-16 events). Extremely dense bursts of data might be over-summarized.
- Causality in Sampling: The paper notes that in clinical settings, the decision to take a measurement is often a signal in itself (e.g., a doctor orders a test because the patient looks worse). TRESS models the values but doesn't explicitly model the "policy" behind the sampling.
- Future Directions: The authors suggest exploring how this compact representation could be used in streaming applications or integrated into larger foundation models for time series.
Summary updated