MMNet: Missing-Aware and Memory-Enhanced Network for Multivariate Time Series Imputation

Xiaoye Miao, Han Shi, Yi Yuan, Daozhan Pan, Yangyang Wu, Xiaohua PanIJCAI 2025 · DOI

TL;DR

This paper introduces MMNet, a novel deep learning framework for imputing missing values in multivariate time series data. MMNet addresses limitations of existing methods by adaptively learning to distinguish between observed and missing data and by incorporating global temporal patterns through a memory-enhanced encoder, leading to significant performance gains.

What Problem Does This Paper Address?

Multivariate time series (MTS) data is ubiquitous in real-world applications, from healthcare to energy monitoring. However, this data is often incomplete due to sensor failures, environmental issues, or privacy concerns. Imputing these missing values is crucial for accurate analysis and downstream tasks. Traditional imputation methods struggle with the complex, non-linear temporal dependencies in MTS. While deep learning methods have shown promise, they often initialize missing values with zeros, introducing noise and hindering the model's ability to discern missingness patterns. Furthermore, many existing approaches focus on local information within a time series, neglecting valuable global similarities that exist across different time segments or even entire series. This paper aims to bridge this gap by developing a method that can effectively handle missing data while leveraging both local and global temporal information.

Methods

MMNet is built upon three core modules:

  • Missing-Aware Embedding (MAE): This module tackles the issue of noisy zero-filling. Instead of simply replacing missing values with zeros, MAE uses learnable vectors to represent missing components. This allows the model to adaptively learn and distinguish between observed and missing data points, even when missing patterns vary across different time series. The module also includes a self-learning embedding approach to avoid introducing noise from pre-filled values.
  • Memory-Enhanced Encoder (MEE): To capture global temporal patterns, MEE incorporates a memory mechanism. This allows the model to leverage prior knowledge and global similarities within the time series, going beyond the limitations of local context. It achieves this by integrating learnable memory vectors into the attention computation, enabling the model to retrieve and capture relationships beyond individual samples.
  • Multi-scale Mixing (MSM): Time series data exhibits characteristics at different scales. The MSM architecture extracts features at multiple scales and then progressively merges them. This "coarse-to-fine" mixing strategy allows the model to integrate both macroscopic trends and finer details, leading to more robust imputation. The MEE module is integrated at each scale within the MSM.

The overall framework first processes the input MTS through MAE to get a hidden representation. This representation is then fed into the MSM, which uses MEE at each scale. Finally, a coarse-to-fine mixing strategy combines the multi-scale encodings, and a linear layer imputes the missing values.

Datasets

The paper evaluates MMNet on four publicly available real-world MTS datasets:

  • PhysioNet 2012 Mortality Prediction Challenge Dataset: A large dataset of clinical time series from ICU patients, with a very high missing rate (80.67%).
  • Beijing Multi-site Air Quality Dataset (Air-Quality): Hourly air pollutant data from multiple monitoring stations.
  • Electric Transformer Temperature Dataset (ETT): Specifically, the ETTh1 subset, which records hourly electricity consumption and related features.
  • Italy Air Quality Dataset (Italy-Air): Hourly average response values from sensors and pollutant concentration data.

The paper also mentions using PyGrinder for simulating six different missing mechanisms (MCAR, MCAR_B, MAR, MAR_B, MNAR, MNAR_B) for more comprehensive evaluation.

Key Insights and Ideas

  • Adaptive Missing Representation is Crucial: Simply filling missing values with zeros is detrimental. MMNet's MAE module demonstrates that learning specific embeddings for missing data is key to distinguishing it from observed data and improving imputation accuracy.
  • Global Context Matters: Existing methods often overlook global temporal similarities. The MEE module's memory mechanism effectively captures these long-range dependencies, leading to more informed imputation, especially in sparse data scenarios.
  • Multi-scale Processing Enhances Robustness: Time series have patterns at various resolutions. The MSM architecture's ability to extract and integrate information from multiple scales provides a more comprehensive understanding of the data, leading to better imputation performance.
  • Accurate Imputation Boosts Downstream Tasks: The paper shows that improved imputation quality directly translates to better performance in downstream tasks, such as classification, as demonstrated on the PhysioNet dataset.

Key Results

MMNet significantly outperforms eight state-of-the-art imputation methods across all four datasets and metrics (MAE and MSE).

  • Overall Performance: MMNet achieved an average improvement of 26.87% (MAE) and 28.62% (MSE) over the best baseline method (CSDI). On the Italy-Air dataset, the gains were even more substantial, reaching up to 58.79% (MAE) and 59.52% (MSE).
  • Robustness to Missing Rate: MMNet demonstrated superior robustness to increasing missing rates (from 10% to 90%), maintaining more stable accuracy compared to other methods.
  • Downstream Task Performance: When used to impute data for a downstream classification task on the PhysioNet dataset, MMNet-imputed data led to the best classification results, with improvements of 0.42% (PR-AUC), 1.09% (ROC-AUC), and 0.89% (F1-Score) over the next best method.
  • Ablation Study: The study confirmed that each component (MAE, MEE, MSM) contributes positively to MMNet's performance, with MAE showing the most significant impact.

Limitations and Future Work

  • Computational Cost: While MMNet achieves superior performance, the paper doesn't explicitly detail the computational overhead compared to baselines. The multi-scale processing and memory mechanisms might introduce higher computational demands.
  • Hyperparameter Sensitivity: The paper mentions hyperparameter optimization, but the sensitivity of MMNet to specific hyperparameters (e.g., number of memory vectors, scales) could be further explored.
  • Future Work: The authors suggest that their framework can be extended to other time series tasks beyond imputation, such as forecasting and anomaly detection, by adapting the output layer and loss functions. Further research could also explore more sophisticated memory mechanisms or adaptive multi-scale strategies.

Summary updated

BibTeX
@inproceedings{miao2025mmnet,
  author = {Xiaoye Miao and Han Shi and Yi Yuan and Daozhan Pan and Yangyang Wu and Xiaohua Pan},
  title = {{MMNet: Missing-Aware and Memory-Enhanced Network for Multivariate Time Series Imputation}},
  booktitle = {International Joint Conference on Artificial Intelligence},
  year = {2025},
  doi = {10.24963/ijcai.2025/357},
}
Figure