A Reproducible Comparison of RSSI Fingerprinting Localization Methods Using LoRaWAN

Grigorios G. Anagnostopoulos, Alexandros Kalousis

TL;DR

This paper presents a reproducible comparison of three machine learning methods (kNN, Extra Trees, and MLP) for RSSI fingerprinting localization using LoRaWAN data. The authors found that a Multilayer Perceptron (MLP) achieved the highest accuracy, with a mean error of 358 meters, and made their code and data splits publicly available to foster reproducibility.

What Problem Does This Paper Address?

Locating low-power Internet of Things (IoT) devices in outdoor environments is a growing challenge, especially when these devices cannot afford the power consumption of traditional GPS. Low Power Wide Area Networks (LPWANs) like LoRaWAN offer a promising alternative by using radio frequency (RF) signals. One localization technique is RSSI fingerprinting, which relies on matching current signal strength measurements to a pre-existing database of signal fingerprints. However, comparing different machine learning approaches for this task, especially with LPWANs, requires standardized datasets and reproducible methods. This paper addresses this by providing a direct, reproducible comparison of common machine learning algorithms on a publicly available LoRaWAN RSSI dataset.

Methods

The authors evaluated three distinct machine learning algorithms for RSSI fingerprinting:

  • k-Nearest Neighbors (kNN): A classic and widely used method that finds the 'k' closest training data points to a new measurement and predicts the location based on their average.
  • Extra Trees: An ensemble method based on decision trees, which introduces more randomness in tree construction compared to Random Forests.
  • Multilayer Perceptron (MLP): A type of feed-forward artificial neural network, used here as a deep learning approach.

They also explored four different data preprocessing schemes (positive, normalized, exponential, and powed representations) to transform the raw RSSI values. For each method, they performed hyperparameter tuning and evaluated various distance metrics to find the optimal configuration. Crucially, they made their code and the train/validation/test splits of the dataset publicly available to ensure reproducibility.

Datasets

The study utilized a publicly available LoRaWAN RSSI dataset collected in the urban area of Antwerp, Belgium, originally published by Aernouts et al. [2]. This dataset contains RSSI measurements from 68 basestations, along with GPS ground truth locations. The authors split this dataset into training (70%), validation (15%), and testing (15%) sets, making these splits available for future research.

Key Insights and Ideas

  • MLP Outperforms Simpler Models: The neural network (MLP) demonstrated superior performance compared to kNN and Extra Trees, suggesting that complex models can better capture the nuances of RSSI signal propagation in an outdoor LoRaWAN environment.
  • Dataset Sparsity is a Challenge: The dataset is noted to be quite sparse, with most signal transmissions received by only three basestations. This sparsity makes accurate localization a challenging task, as it limits the richness of the fingerprint data.
  • Preprocessing Matters, But Less Than Expected: While various preprocessing techniques were explored, the differences in performance among them were not as significant as might be expected. This is attributed to the dataset's sparsity, where the common set of receiving basestations appears to be a more dominant factor in determining fingerprint similarity than the specific RSSI transformation.
  • Reproducibility is Key: The authors emphasize the importance of making code and data splits available, which is crucial for validating and building upon research in this area.

Key Results

The study found the following performance metrics on the test set:

  • kNN: Achieved a mean error of 394 meters and a median error of 273 meters.
  • Extra Trees: Achieved a mean error of 380 meters and a median error of 261 meters.
  • MLP: Achieved the best performance with a mean error of 358 meters and a median error of 204 meters.

The MLP's performance represents a significant improvement over previous work using the same dataset, particularly in median error.

Limitations and Future Work

  • Dataset Limitations: The authors acknowledge that the ground truth is determined by GPS estimates, which can have inaccuracies. The sparsity of the dataset is also highlighted as a significant challenge.
  • Hyperparameter Tuning: While efforts were made to tune hyperparameters, an exhaustive search for all methods was beyond the scope of the study.
  • Future Work: The authors suggest that further research could explore more advanced deep learning architectures and investigate how different preprocessing schemes might interact with these more complex models. They also implicitly suggest that exploring methods to mitigate the impact of dataset sparsity could be beneficial.

Summary updated

Abstract

The use of fingerprinting localization techniques in outdoor IoT settings has started gaining popularity over the recent years. Communication signals of Low Power Wide Area Networks (LPWAN), such as LoRaWAN, are used to estimate the location of low power mobile devices. In this study, a publicly available dataset of LoRaWAN RSSI measurements is utilized to compare different machine learning methods and their accuracy in producing location estimates. The tested methods are: the k Nearest Neighbours method, the Extra Trees method and a neural network approach using a Multilayer Perceptron. To facilitate the reproducibility of tests and the comparability of results, the code and the train/validation/test split of the dataset used in this study have become available. The neural network approach was the method with the highest accuracy, achieving a mean error of 357 meters and a median error of 206 meters.
BibTeX
@inproceedings{anagnostopoulos2019reproducible,
  author = {Grigorios G. Anagnostopoulos and Alexandros Kalousis},
  title = {{A Reproducible Comparison of RSSI Fingerprinting Localization Methods Using LoRaWAN}},
  booktitle = {WPNC},
  year = {2019},
  url = {https://openreview.net/forum?id=m5eyRGDE86F},
}

Loading related content...

Figure