Scene Recognition for Indoor Localization Using a Multi-Sensor Fusion Approach

Mengyun Liu, Ruizhi Chen, Deren Li, Yujin Chen, Guangyi Guo, Zhipeng Cao, Yuanjin Pan2017 · DOI

TL;DR

This paper tackles the persistent challenge of accurate indoor localization by proposing a novel approach that fuses multi-sensor data (WiFi, magnetic fields, inertial sensors) with scene recognition powered by deep learning. By leveraging visual scene context, the system significantly improves localization accuracy and robustness, outperforming existing commercial solutions.

What Problem Does This Paper Address?

Despite decades of research, a reliable and ubiquitous indoor localization solution comparable to outdoor GPS remains elusive. This is largely due to the complex, multi-path RF environments and intricate spatial layouts of indoor spaces. Existing fingerprinting-based methods, while effective, suffer from labor-intensive data collection, difficult database maintenance, signal ambiguities, and slow matching. This paper aims to overcome these limitations by incorporating human-like visual cognition into the localization process.

Methods

The proposed system employs a multi-sensor fusion approach, integrating data from a smartphone's camera, WiFi, and inertial sensors (accelerometer, gyroscope, compass, magnetometer). The core innovation lies in using deep learning (specifically, a fine-tuned CaffeNet architecture) for indoor scene recognition. This scene information then acts as a constraint, narrowing down the search space within a particle filter-based localization framework.

Here's a breakdown of the key methods:

  • Data Acquisition: Captures images, WiFi RSSI, magnetic field strength (MFS), and inertial sensor data (for step detection and heading) while users walk.
  • Indoor Scene Recognition: A deep convolutional neural network (CNN), fine-tuned from a pre-trained ImageNet model, is used to classify the current indoor scene from camera images.
  • Fingerprint Database: Constructed using user trajectories, organized by scene labels and transition points (TPs). TPs serve as robust reference points.
  • Particle Filter Localization: Fuses WiFi and MFS signals to update particle weights. The key innovation here is that the particle filter operates within the sub-fingerprint database corresponding to the recognized scene, significantly reducing search space and ambiguity.
  • Transition Point Detection: A k-NN algorithm is used to detect transition points, which are crucial for initiating localization and resetting the PDR (Pedestrian Dead Reckoning) process when scene changes occur.
  • Sensor Fusion: A weighted fusion approach combines magnetic and WiFi signal distances, with magnetic signals given higher weight due to their generally higher stability and less susceptibility to noise in certain environments.

Datasets

The paper utilizes a custom dataset collected within a laboratory environment spanning two floors. This dataset includes images, WiFi RSSI, magnetic field strength, and inertial sensor data. The authors do not explicitly mention using publicly available indoor localization datasets for training or evaluation, focusing instead on their self-collected data.

Key Insights and Ideas

  • Visual Context is Key: The paper strongly advocates for the importance of visual scene information in indoor localization, drawing a parallel to human spatial cognition. This "seeing" capability significantly enhances localization accuracy.
  • Deep Learning for Scene Understanding: Leveraging deep learning for scene recognition is shown to be highly effective, enabling the system to understand complex visual environments without manual feature engineering.
  • Scene as a Constraint: Using the recognized scene as a constraint for the particle filter dramatically reduces the search space and mitigates signal ambiguities, a major bottleneck in traditional methods.
  • Crowdsourcing Potential: The data acquisition method, based on user trajectories, naturally lends itself to a crowdsourcing approach for building and expanding the fingerprint database, addressing the labor-intensive data collection problem.
  • Hybrid Approach Advantage: The fusion of multiple sensor modalities (WiFi, magnetic, inertial) with scene recognition proves more robust than relying on any single sensor or even combinations without contextual information.

Key Results

  • Recognition Accuracy: The fine-tuned deep learning model achieved a validation accuracy of 97.5% and a test accuracy of 89.8% for indoor scene recognition.
  • Localization Accuracy: The proposed system achieved a 95% localization accuracy of 1.32 m.
  • Performance Comparison:
    • The proposed system (PDR + WiFi + Magnetic + Indoor Scene) achieved a mean error of 0.53 m and 95% accuracy of 1.32 m.
  • A trajectory-based method without indoor scene constraint had a mean error of 2.36 m and 95% accuracy of 5.93 m.
  • The commercial product IndoorAtlas achieved a mean error of 1.08 m and 95% accuracy of 3.01 m.
  • The proposed system significantly outperformed both the trajectory-based method without scene constraint and the commercial IndoorAtlas solution.

Limitations and Future Work

  • Scene Recognition Robustness: The authors acknowledge that scene recognition may not perform well in dark environments or under poor visual conditions. They offer the option to disable the scene recognition module in such cases.
  • Device Heterogeneity and Data Integration: For future crowdsourcing efforts, addressing challenges related to device heterogeneity and integrating data from various sources will be crucial.
  • Efficient Network Architecture: Developing more efficient network architectures for indoor scene recognition and extending the system to handle more complex environments are suggested future research directions.
  • Model Training: While fine-tuning reduces the need for massive datasets, training deep learning models from scratch is still time-consuming. Future work could explore more efficient training strategies.

Summary updated

BibTeX
@misc{liu2017scene,
  author = {Mengyun Liu and Ruizhi Chen and Deren Li and Yujin Chen and Guangyi Guo and Zhipeng Cao and Yuanjin Pan},
  title = {{Scene Recognition for Indoor Localization Using a Multi-Sensor Fusion Approach}},
  year = {2017},
  doi = {10.3390/s17122847},
}
Figure