Prediction of Pollutant Concentration Based on Spatial–Temporal Attention, ResNet and ConvLSTM

Cai Chen, Agen Qiu, Haoyu Chen, Yajun Chen, Xu Liu, Dong LiSensors 2023 · DOI

TL;DR

This paper introduces STA-ResConvLSTM, a novel deep learning model that significantly improves air pollutant concentration prediction by effectively capturing complex spatial and temporal relationships. By integrating spatial-temporal attention mechanisms with ResNet and ConvLSTM, the model achieves state-of-the-art accuracy in both short-term and long-term forecasting, crucial for public health and environmental management.

What Problem Does This Paper Address?

Accurate prediction of air pollutant concentrations, particularly PM2.5, is vital for public health advisories and environmental policy. However, existing models struggle due to the inherent complexity of air pollution, which is influenced by dynamic pollution sources, changing meteorological conditions, and intricate atmospheric transformations. These factors create significant uncertainty and instability, making it difficult for traditional models to effectively extract the crucial spatial and temporal correlations within the data. This paper aims to bridge this gap by developing a more robust prediction model capable of handling these challenges.

Methods

The core of this work is the STA-ResConvLSTM model, a hybrid deep learning architecture. It's built upon three key components:

  • ResNet (Residual Neural Network): Used as the foundational layer to extract spatial features from meteorological and pollution data. ResNet's architecture helps mitigate the vanishing gradient problem in deep networks, allowing for more effective feature extraction.
  • Spatial-Temporal Attention Mechanism: This novel component is embedded within each residual unit of the ResNet. It's designed to intelligently focus on the most relevant spatial and temporal aspects of the input data, effectively filtering out noise and highlighting critical information. This mechanism combines both spatial attention (focusing on relationships across different locations) and channel attention (focusing on relationships across different pollutant types and meteorological factors).
  • ConvLSTM (Convolutional Long Short-Term Memory): This layer takes the output from the STA-ResNet and further processes it to capture intricate spatial-temporal correlations. ConvLSTM combines the strengths of Convolutional Neural Networks (CNNs) for spatial feature extraction and LSTMs for temporal sequence modeling, making it ideal for this task.

The overall framework first uses STA-ResNet to extract high-level spatial semantic characteristics, which are then fed into ConvLSTM for detailed spatial-temporal correlation feature extraction and final pollutant concentration prediction.

Datasets

The study utilizes historical air pollutant concentration data and meteorological data for Beijing and its neighboring urban agglomerations.

  • Air Pollutant Data: Hourly data for Air Quality Index (AQI), PM2.5, PM10, O3, NO2, CO, and SO2 were obtained from the Ministry of Ecology and Environment of the People's Republic of China.
  • Meteorological Data: Hourly data for temperature, humidity, and wind speed were sourced from the Public Weather website.

The data spans the year 2021 and was preprocessed using simple linear interpolation for missing values and Min-Max normalization. The dataset was split into training (70%), validation (20%), and testing (10%) sets.

Key Insights and Ideas

  • The Power of Hybrid Models: The research strongly suggests that combining different deep learning architectures (like CNNs with LSTMs, or ResNet with ConvLSTM) significantly outperforms single-architecture models for complex time-series prediction tasks like air pollution forecasting.
  • Attention is Key: The integration of spatial-temporal attention mechanisms is crucial. It allows the model to dynamically weigh the importance of different spatial locations and time steps, leading to more accurate and robust predictions, especially in capturing abrupt changes.
  • Spatial-Temporal Interdependence: The study highlights that air pollutant concentrations are not just a function of past values but are deeply influenced by both their spatial distribution across regions and their temporal evolution. The proposed model effectively leverages this interdependence.
  • ResNet's Role in Feature Extraction: Using ResNet as a base helps in extracting deeper spatial features from diverse data sources (multiple cities, various pollutants, and meteorological factors) without succumbing to the vanishing gradient problem.

Key Results

The STA-ResConvLSTM model demonstrated superior performance across various prediction tasks:

  • Single-Step Prediction (1-hour ahead): Achieved a Root Mean Square Error (RMSE) of 9.82 and a Mean Absolute Error (MAE) of 5.86, with an R² of 0.9307 and an Index of Agreement (IA) of 98.29%. This significantly outperformed baseline models like CNN, LSTM, CNN-LSTM, and ConvLSTM.
  • Multi-Step Prediction (1-48 hours ahead): The model maintained strong performance, achieving an average RMSE of 13.49. Even for longer prediction horizons (up to 48 hours), STA-ResConvLSTM showed less fluctuation in accuracy compared to other models, indicating its stability and reliability.
  • Trend Prediction: The model accurately captured the trend of PM2.5 concentration, including sudden change points, outperforming other models in fitting observed values.

Limitations and Future Work

  • Data Distribution: The authors acknowledge that an unequal distribution of data, particularly for mutation points (sudden changes in pollutant concentration), can pose challenges for model learning and lead to difficulties in fitting abrupt increases.
  • Future Work:
    • Grid-based Data Processing: To further enhance spatial information extraction, the authors suggest dividing pollutant and meteorological data from different cities into grids.
  • Expanded Data Scope: Incorporating data from more cities or longer time spans is proposed to potentially increase prediction accuracy.
  • Model Refinement: While the STA-ResConvLSTM model is effective, further research into combining it with other state-of-the-art prediction methods could lead to even greater accuracy.

Summary updated

Abstract

Accurate and reliable prediction of air pollutant concentrations is important for rational avoidance of air pollution events and government policy responses. However, due to the mobility and dynamics of pollution sources, meteorological conditions, and transformation processes, pollutant concentration predictions are characterized by great uncertainty and instability, making it difficult for existing prediction models to effectively extract spatial and temporal correlations. In this paper, a powerful pollutant prediction model (STA-ResConvLSTM) is proposed to achieve accurate prediction of pollutant concentrations. The model consists of a deep learning network model based on a residual neural network (ResNet), a spatial-temporal attention mechanism, and a convolutional long short-term memory neural network (ConvLSTM). The spatial-temporal attention mechanism is embedded in each residual unit of the ResNet to form a new residual neural network with the spatial-temporal attention mechanism (STA-ResNet). Deep extraction of spatial-temporal distribution features of pollutant concentrations and meteorological data from several cities is carried out using STA-ResNet. Its output is used as an input to the ConvLSTM, which is further analyzed to extract preliminary spatial-temporal distribution features extracted from the STA-ResNet. The model realizes the spatial-temporal correlation of the extracted feature sequences to accurately predict pollutant concentrations in the future. In addition, experimental studies on urban agglomerations around Long Beijing show that the prediction model outperforms various popular baseline models in terms of accuracy and stability. For the single-step prediction task, the proposed pollutant concentration prediction model performs well, exhibiting a root-mean-square error (RMSE) of 9.82. Furthermore, even for the pollutant prediction task of 1 to 48 h, we performed a multi-step prediction and achieved a satisfactory performance, being able to achieve an average RMSE value of 13.49.
BibTeX
@inproceedings{chen2023prediction,
  author = {Cai Chen and Agen Qiu and Haoyu Chen and Yajun Chen and Xu Liu and Dong Li},
  title = {{Prediction of Pollutant Concentration Based on Spatial–Temporal Attention, ResNet and ConvLSTM}},
  booktitle = {Sensors},
  year = {2023},
  doi = {10.3390/s23218863},
}
Figure