Air Pollution Modelling by Machine Learning Methods

Petra Vidnerová, Roman NerudaModelling—International 2021 · DOI

TL;DR

This paper tackles the challenge of accurate urban air pollution monitoring by proposing a machine learning approach that fuses data from numerous small, inexpensive, but less reliable sensors. They demonstrate that even with these limitations, sophisticated machine learning models, particularly deep neural networks and custom kernel methods, can achieve reliable pollutant concentration predictions, offering a more scalable solution than traditional sparse monitoring networks.

What Problem Does This Paper Address?

Urban air pollution is a critical environmental and health concern, necessitating precise monitoring to inform traffic management and urban planning. Current monitoring relies on sparse networks of fixed, high-quality stations, which struggle to capture the complex spatial distribution of pollutants in urban environments. The authors aim to overcome this limitation by leveraging the potential of deploying a massive number of small, low-cost, multi-sensor devices. While these sensors are affordable and can be deployed widely, their individual readings are less reliable. The core problem is how to effectively fuse and process the data from these numerous, potentially noisy sensors to produce accurate estimates of pollutant concentrations (CO, NO2, and NOx).

Methods

The paper explores several machine learning techniques for sensor fusion and pollutant prediction. The primary methods investigated are:

  • Regularization Networks (RNs): A type of feed-forward neural network with a single hidden layer, grounded in a strong theoretical framework for supervised learning.
  • RNs with Composite Kernels: An extension of RNs where the kernel function, which defines the similarity between data points, is a combination of simpler kernel functions (e.g., Gaussian, polynomial). This allows for more flexible modeling of complex data relationships. The authors propose original composite kernel functions and evolutionary algorithms to search for optimal kernel combinations.
  • Deep Neural Networks (DNNs): Networks with multiple hidden layers, which have become state-of-the-art in many machine learning domains.
  • Support Vector Regression (SVR): A standard regression technique used for comparison.

A significant methodological contribution is the extensive hyper-parameter search for all models. For RNs and their composite kernels, a genetic algorithm (GA) was employed. For DNNs, evolution strategies (ES) were used to optimize the network architecture (number of layers, neurons per layer, activation functions). This rigorous search process aims to ensure a fair comparison by finding the best possible configuration for each model.

Datasets

The study utilizes a real-world dataset previously used in related research [7, 8]. This dataset comprises measurements from a multi-sensor MOX array device, recording various gas pollutant concentrations. The data is labeled by a conventional air pollution monitoring station. Key characteristics of the dataset include:

  • Frequency: 1-hour measurements.
  • Missing Values: The data contains many missing values due to sensor malfunctions, which were handled by neglecting lines with missing data.
  • Inputs: Five sensor measurements.
  • Outputs (Targets): Concentrations of CO, NO2, and NOx gases.
  • Timeframe: Data from March 10, 2004, to April 4, 2005.

Two main experimental setups were used:

  1. Sparse Measurements (Task 1): A simplified scenario using 4 data samples per day for training and the rest for testing, focusing on predicting short intervals.
  2. Uniform Intervals (Task 2): A more realistic scenario where the data was divided into five uniform intervals. In each sub-experiment, one interval was used for training, and the remaining 80% for testing, simulating prediction across different seasons.

Key Insights and Ideas

  • The Power of Sensor Fusion: The paper strongly supports the idea that combining data from numerous, even individually unreliable, sensors can lead to accurate environmental predictions. This opens up possibilities for more cost-effective and widespread pollution monitoring.
  • Composite Kernels for Flexibility: The authors' work on composite kernels for Regularization Networks demonstrates their ability to capture complex data patterns more effectively than simple kernels, especially when dealing with multi-modal data.
  • Evolutionary Optimization for Hyper-parameters: Employing genetic algorithms and evolution strategies for hyper-parameter tuning (kernel selection, network architecture) is crucial for achieving optimal performance and ensuring a fair comparison between different model types. This automated approach is vital for complex models.
  • DNNs Excel at Generalization: While product kernels showed excellent fitting on training data, Deep Neural Networks consistently demonstrated superior generalization ability, performing better on unseen test data, particularly in more challenging scenarios.
  • NO2 is the Toughest Pollutant: The study consistently found that predicting NO2 concentrations yielded the highest errors across most models, highlighting it as a particularly challenging pollutant to model accurately.

Key Results

  • All tested models performed satisfactorily: The study found that all machine learning models investigated were capable of modeling pollutant concentrations from multi-sensor data.
  • Product Kernels for Training Fit: Regularization Networks with product kernels achieved the best fitting on the training data, indicating their ability to closely learn the provided training examples.
  • DNNs for Generalization: Deep Neural Networks exhibited the best generalization ability, performing better on test sets, especially in the more complex, seasonal prediction task.
  • RNs Outperform SVR: In general, Regularization Networks (both Gaussian and product kernels) provided better results than Support Vector Regression models.
  • Task 1 Performance: For the simpler task of predicting short intervals, product kernels performed best on both training and testing errors.
  • Task 2 Performance: In the more challenging seasonal prediction task, product kernels excelled in training errors, while DNNs showed better performance in testing errors, indicating their superior generalization.

Limitations and Future Work

  • Missing Data Handling: The paper acknowledges that the current work did not specifically address the issue of missing data, which is a significant challenge in real-world sensor datasets. They suggest that simple mean imputation is insufficient and recommend more sophisticated methods like clustering or semi-supervised techniques.
  • Dataset Size and Specificity: While the dataset used is described as representative, the authors note that it is relatively small. They suggest that for practical deployment, especially considering seasonal variations in pollution, ensemble techniques combining models trained on different seasons would be beneficial.
  • Generalization to Other Pollutants/Environments: The study focused on specific pollutants (CO, NO2, NOx) and a particular urban environment. Further research could explore the applicability of these methods to other pollutants and diverse geographical settings.
  • Ensemble Methods: The authors explicitly suggest that ensemble techniques could improve robustness and performance, particularly when dealing with seasonal variations.
  • Advanced Data Preprocessing: The paper highlights the importance of better data preprocessing and cleaning for improved results.

Summary updated

Abstract

Precise environmental modelling of pollutants distributions represents a key factor for addresing the issue of urban air pollution. Nowadays, urban air pollution monitoring is primarily carried out by employing sparse networks of spatially distributed fixed stations. The work in this paper aims at improving the situation by utilizing machine learning models to process the outputs of multi-sensor devices that are small, cheap, albeit less reliable, thus a massive urban deployment of those devices is possible. The main contribution of the paper is the design of a mathematical model providing sensor fusion to extract the information and transform it into the desired pollutant concentrations. Multi-sensor outputs are used as input information for a particular machine learning model trained to produce the CO, NO2, and NOx concentration estimates. Several state-of-the-art machine learning methods, including original algorithms proposed by the authors, are utilized in this study: kernel methods, regularization networks, regularization networks with composite kernels, and deep neural networks. All methods are augmented with a proper hyper-parameter search to achieve the optimal performance for each model. All the methods considered achieved vital results, deep neural networks exhibited the best generalization ability, and regularization networks with product kernels achieved the best fitting of the training set.
BibTeX
@inproceedings{vidnerova2021air,
  author = {Petra Vidnerová and Roman Neruda},
  title = {{Air Pollution Modelling by Machine Learning Methods}},
  booktitle = {Modelling—International},
  year = {2021},
  doi = {10.3390/modelling2040035},
}
Figure