General Incomplete Time Series Analysis via Patch Dropping Without Imputation

Yangyang Wu, Yi Yuan, Mengying Zhu, Xiaoye Miao, Meng XiIJCAI 2025 · DOI

TL;DR

This paper introduces INTER, a novel end-to-end framework for analyzing incomplete multivariate time series data that bypasses traditional imputation methods. By leveraging pre-trained language models and a new "patch dropping" strategy, INTER significantly improves accuracy and robustness on various time series tasks without the pitfalls of imputation.

What Problem Does This Paper Address?

Multivariate time series analysis is crucial for many real-world applications, from predicting stock prices to monitoring patient health. However, real-world data is often incomplete due to sensor failures or environmental issues, with some datasets having over 80% missing values. Existing methods struggle with this incompleteness: some ignore missing data, leading to poor performance, while others use an "impute-then-analyze" approach. This latter method suffers from redundant training and error accumulation during imputation, resulting in biased and suboptimal analyses. This paper tackles the challenge of effectively analyzing time series data with missing values without resorting to imputation.

Methods

The proposed framework, INTER, consists of two main modules: Missing-aware Patch Learning (MPL) and Incomplete Time Series Analysis (ITSA).

  • MPL Module:
    • TSE (Incomplete Time Series Encoding): Extracts missing state information and generates initial embeddings of the time series. It uses a feedforward neural network to learn a "missing state matrix" and applies Z-score scaling and input embedding to prepare the data.
  • MPD (Missing-state-aware Patch Dropping): This novel strategy dynamically drops incomplete time series patches based on their missing rate. Patches with more missing data are more likely to be dropped, aiming to reduce noise and improve robustness. The authors theoretically prove that this method results in lower sample variance compared to traditional dropout strategies.
  • MAT (Missing-aware Transformer): Processes the patched time series data, incorporating the missing state information. It uses a modified attention mechanism to exclude dropped patches, focusing on valid data and learning temporal dependencies.
  • ITSA Module:
    • Leverages pre-trained language models (PLMs), specifically GPT-2, to provide prior knowledge for analyzing incomplete time series.
  • The MPL module's output is treated as a sequence of tokens for the PLM.
  • A novel element-wise loss function is introduced to mitigate the risk of learning incorrect information from missing values and to improve performance on various downstream tasks like forecasting, classification, and anomaly detection.

Datasets

The paper evaluates INTER on 11 public real-world multivariate time series datasets across various domains, including:

  • Electricity
  • Weather
  • Exchange
  • Illness
  • PhysioNet (medical dataset)
  • Heartbeat (classification)
  • Japanese Vowels (classification)
  • M4 dataset (yearly and monthly subsets for short-term forecasting)

Key Insights and Ideas

  • Bypassing Imputation is Key: The core idea is that directly analyzing incomplete data, rather than imputing and then analyzing, can avoid error propagation and lead to more robust and accurate results.
  • Dynamic Patch Dropping: The MPD strategy is a significant innovation. Instead of uniform dropout, it intelligently drops patches based on their missingness, acting as a form of regularization that specifically targets noisy or incomplete segments.
  • Leveraging PLMs for Time Series: Applying pre-trained language models, typically used for text, to time series data is a powerful transfer learning approach. It allows the model to benefit from general patterns learned from vast amounts of data.
  • Theoretical Justification: The theoretical proof for MPD's lower sample variance adds significant credibility to the proposed method, suggesting a principled approach to handling missing data.

Key Results

  • Significant Accuracy Improvement: INTER improves accuracy by over 20% compared to state-of-the-art methods across 11 datasets for long-term forecasting. For example, it achieved a 20.91% average improvement in MSE over the best imputation-based method (Timer with SAITS) and a 76.20% gain over TriD-MAE.
  • Strong Performance Across Tasks: INTER demonstrates superior performance in long-term forecasting, short-term forecasting, and classification tasks.
  • Competitive Computational Efficiency: While achieving better performance, INTER maintains comparable computational efficiency to existing methods.
  • Robustness: The MPD strategy contributes to enhanced model generalization and robustness, particularly in the presence of high missing rates.

Limitations and Future Work

  • Computational Cost of PLMs: While effective, the use of large pre-trained language models can be computationally intensive. The paper mentions freezing most PLM layers during fine-tuning to mitigate this.
  • Generalization to Other Tasks: The paper focuses on forecasting, classification, and anomaly detection. While the framework is designed to be generalizable, further exploration of its effectiveness on other time series tasks (e.g., imputation itself) might be beneficial.
  • Theoretical Analysis Scope: The theoretical proof for MPD is specific to certain dropping strategies and distributions. Expanding this analysis to cover a broader range of scenarios could be valuable.

The authors suggest that INTER's ability to directly learn from incomplete data and its novel components make it a promising direction for future research in time series analysis.

Summary updated

BibTeX
@inproceedings{wu2025general,
  author = {Yangyang Wu and Yi Yuan and Mengying Zhu and Xiaoye Miao and Meng Xi},
  title = {{General Incomplete Time Series Analysis via Patch Dropping Without Imputation}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.24963/ijcai.2025/734},
}
Figure