Missing Value Imputation of Wireless Sensor Data for Environmental Monitoring

Thomas Decorte, Steven Mortier, Jonas J. Lembrechts, Filip J. R. Meysman, Steven Latré, Erik Mannens, Tim Verdonck

TL;DR

This paper tackles the pervasive problem of missing data in large-scale wireless sensor networks (WSNs) used for environmental monitoring. By evaluating 12 imputation methods on a real-world dataset from a citizen science project, they found that methods leveraging spatial correlations, particularly Matrix Completion, significantly outperform time-series-based approaches, offering a crucial tool for maximizing the value of expensive sensor networks.

What Problem Does This Paper Address?

Wireless Sensor Networks (WSNs) are increasingly deployed for environmental monitoring, generating vast amounts of spatiotemporal data. However, these networks are prone to data loss due to sensor failures, communication errors, or even vandalism. This missing data can severely hamper data analysis, leading to biased conclusions, inaccurate predictions, and delayed responses, especially in critical applications like environmental monitoring. While various imputation techniques exist, their effectiveness on large-scale, real-world WSN datasets, particularly those with complex missing patterns, remains an open question. This paper aims to bridge this gap by comprehensively evaluating a range of imputation methods on a substantial environmental monitoring dataset.

Methods

The researchers evaluated 12 different imputation methods, categorizing them by their approach:

  • Temporal: Methods that use a sensor's own past data (e.g., Mean Imputation, Spline Imputation).
  • Spatial: Methods that leverage data from neighboring sensors (e.g., KNN Imputation, MICE, MCMC, MissForest).
  • Combined/WSN-specific: Methods designed to handle both temporal and spatial correlations, or specifically for WSNs (e.g., DESM, AKE).
  • Deep Learning: Advanced methods utilizing neural networks (e.g., M-RNN, MIDA, BRITS).

They also introduced two distinct missing data scenarios for evaluation:

  1. Artificial Random Missingness: Data points were randomly removed at varying percentages (10% to 50%) to simulate Missing Completely at Random (MCAR) scenarios, a common benchmark in imputation research.
  2. Masked Missingness: This more realistic scenario involved using actual missing patterns from incomplete sensors and imposing them onto complete sensors. This approach aims to mimic real-world issues like network outages where multiple sensors might fail simultaneously (block missingness).

Performance was assessed using Root-Mean-Square Error (RMSE) and Mean Absolute Error (MAE) for accuracy, and Prediction Coverage Error (PCE) to measure the proportion of successfully imputed values.

Datasets

The primary dataset used in this study originated from the Curieuze-Neuzen in de Tuin (CNidT) citizen science project. This project deployed IoT-based microclimate sensors (measuring temperature and soil moisture) in 4400 gardens across Flanders, Belgium, collecting data every 15 minutes for six months. The study utilized a subset of this data, specifically temperature readings from sensors at 12 cm above the soil surface, chosen for their higher variability. This dataset is a component of the larger, publicly available SoilTemp database. The researchers also created artificial missing data patterns on a subset of 1500 complete sensor time series for their evaluation.

Key Insights and Ideas

  • Spatial is Superior: The most significant finding is that methods leveraging spatial correlations consistently outperform those relying solely on temporal information. This highlights the importance of inter-sensor relationships in environmental monitoring WSNs.
  • Matrix Completion Reigns Supreme: Among all evaluated methods, Matrix Completion (MC) demonstrated the best performance across both random and masked missing scenarios, achieving the lowest RMSE and MAE. This suggests that exploiting the low-rank structure of the spatiotemporal data matrix is highly effective.
  • Realistic Missingness is Crucial: The performance differences between random and masked missing scenarios underscore the importance of evaluating imputation methods on realistic data patterns. Methods that struggle with block missingness (like Spline and MICE) were significantly impacted in the masked scenario.
  • Deep Learning Falls Short (in this context): Surprisingly, deep learning-based methods (M-RNN, MIDA, BRITS) performed poorly on this specific dataset. The authors attribute this to the relatively short time series and the localized nature of microclimate data, which may not provide enough complex temporal patterns for these models to learn effectively.
  • WSN-Specific Methods Show Promise: While not always outperforming the top general methods, WSN-specific techniques like AKE and DESM showed good performance, though they sometimes failed to impute all values in the masked scenario due to issues like network-wide outages.

Key Results

  • Matrix Completion (MC) consistently achieved the lowest RMSE and MAE across all tested missing percentages (10-50%) and in the masked missing scenario. For example, in the masked scenario, MC achieved an RMSE of 0.493 and MAE of 0.341.
  • MissForest and MCMC also performed well, generally outperforming other methods besides MC.
  • Time-series methods like Mean Imputation and Spline Imputation showed significantly worse performance, especially in the masked missing scenario (e.g., Spline's MAE reached 2.520 in the masked scenario).
  • Deep learning methods like MIDA showed a dramatic increase in error with higher missing percentages, with an MAE of 1.019 in the masked scenario, and M-RNN had a relatively stable but generally poorer performance.
  • AKE and DESM had high Prediction Coverage Error (PCE) in the masked scenario (95.3% and 98.5% respectively), indicating they couldn't impute all missing values, likely due to simultaneous sensor failures.
  • Execution Time: Matrix Completion was notably fast, achieving a very small execution time (2.2 seconds for masked missing) while providing the best results. Deep learning methods were the most computationally intensive.

Limitations and Future Work

  • Dataset Specificity: The authors acknowledge that the poor performance of deep learning methods might be specific to the characteristics of their microclimate dataset (short time series, localized effects). They suggest that these methods might perform better on datasets with different properties.
  • Masked Missingness Nuances: While masked missingness is more realistic, it still might not perfectly capture all real-world scenarios. The authors suggest that future work could explore other complex missingness patterns.
  • Future Directions:
    • Evaluate more imputation methods and different datasets with varying numbers of sensors and temporal granularities.
  • Incorporate additional sensor features or measured variables (e.g., humidity) into imputation models.
  • Develop novel WSN-specific methods that efficiently combine temporal and spatial imputation strategies (e.g., sequential imputation).
  • Investigate cost-sensitive imputation methods that penalize over/underestimations differently.
  • Analyze imputation performance at different temporal resolutions (e.g., hourly vs. 15-min intervals).

Summary updated

Abstract

Over the past few years, the scale of sensor networks has greatly expanded. This generates extended spatiotemporal datasets, which form a crucial information resource in numerous fields, ranging from sports and healthcare to environmental science and surveillance. Unfortunately, these datasets often contain missing values due to systematic or inadvertent sensor misoperation. This incompleteness hampers the subsequent data analysis, yet addressing these missing observations forms a challenging problem. This is especially the case when both the temporal correlation of timestamps within a single sensor and the spatial correlation between sensors are important. Here, we apply and evaluate 12 imputation methods to complete the missing values in a dataset originating from large-scale environmental monitoring. As part of a large citizen science project, IoT-based microclimate sensors were deployed for six months in 4400 gardens across the region of Flanders, generating 15-min recordings of temperature and soil moisture. Methods based on spatial recovery as well as time-based imputation were evaluated, including Spline Interpolation, MissForest, MICE, MCMC, M-RNN, BRITS, and others. The performance of these imputation methods was evaluated for different proportions of missing data (ranging from 10% to 50%), as well as a realistic missing value scenario. Techniques leveraging the spatial features of the data tend to outperform the time-based methods, with matrix completion techniques providing the best performance. Our results therefore provide a tool to maximize the benefit from costly, large-scale environmental monitoring efforts.
BibTeX
@inproceedings{decorte2024missing,
  author = {Thomas Decorte and Steven Mortier and Jonas J. Lembrechts and Filip J. R. Meysman and Steven Latré and Erik Mannens and Tim Verdonck},
  title = {{Missing Value Imputation of Wireless Sensor Data for Environmental Monitoring}},
  booktitle = {Sensors},
  year = {2024},
  url = {https://openreview.net/forum?id=WofoDH5NJw},
}
Figure