Fusion-DHL: WiFi, IMU, and Floorplan Fusion for Dense History of Locations in Indoor Environments

Sachini Herath, Saghar Irandoust, Bowen Chen, Yiming Qian, Pyojin Kim, Yasutaka Furukawa

Abstract

The paper proposes a multi-modal sensor fusion algorithm that fuses WiFi, IMU, and floorplan information to infer an accurate and dense location history in indoor environments. The algorithm uses 1) an inertial navigation algorithm to estimate a relative motion trajectory from IMU sensor data; 2) a WiFi-based localization API in industry to obtain positional constraints and geo-localize the trajectory; and 3) a convolutional neural network to refine the location history to be consistent with the floorplan. We have developed a data acquisition app to build a new dataset with WiFi, IMU, and floorplan data with ground-truth positions at 4 university buildings and 3 shopping malls. Our qualitative and quantitative evaluations demonstrate that the proposed system is able to produce twice as accurate and a few orders of magnitude denser location history than the current standard, while requiring minimal additional energy consumption.

Introduction

Accurate indoor positioning remains a significant challenge because GPS signals are blocked by building materials. Current industry standards, such as the Google Fused Location Provider (FLP), rely heavily on WiFi and Bluetooth "fingerprints." To save battery, these systems often query locations infrequently (e.g., once per minute), resulting in sparse, "jumpy" histories with errors often exceeding 10 meters.

While Inertial Measurement Units (IMUs) in phones can track relative motion at high frequencies, they suffer from "drift"—small errors in sensor readings that accumulate over time, causing the estimated path to veer off-course or "bend" unrealistically. Furthermore, most systems ignore the most obvious constraint: humans cannot walk through walls. This paper addresses the need for a Dense History of Locations (DHL) that is both accurate and consistent with the physical environment.

Overview of the Fusion-DHL system, combining relative trajectories, sparse location data, and floorplans to produce a dense history of location.

Methodology

The authors propose a multi-stage fusion pipeline designed for offline processing, focusing on reconstructing location history rather than real-time navigation. This approach allows the system to use "future" data points to correct "past" errors.

  1. Relative Motion Estimation: The system uses RoNIN, a state-of-the-art deep learning algorithm, to estimate a relative motion trajectory at 50Hz from raw IMU data.
  2. Global Alignment (Optimization): This relative trajectory is "anchored" to the real world using sparse global positions from a WiFi-based API. A non-linear least squares optimization solves for scale and rotational drift, aligning the IMU path with the sparse WiFi points.
  3. Floorplan Refinement (CNN): The aligned trajectory is refined by a Convolutional Neural Network (CNN). The CNN takes a 6-channel input (RGB floorplan + a 3-channel image of the trajectory color-coded by time) and predicts a "correction flow." This flow nudges the trajectory to ensure it stays within walkable areas, avoiding walls or escalators.
  4. Iteration: The system repeats the optimization and CNN steps to converge on a final, high-accuracy path.
The multi-stage fusion pipeline: IMU data processed by RoNIN, WiFi/GPS by Google API, followed by iterative optimization and CNN-based floorplan refinement.

Datasets

A substantial new benchmark for indoor localization was introduced to validate the system:

  • Scale: 15.2 hours of data covering 42.3 kilometers of walking.
  • Diversity: Data was collected across 4 university buildings (training) and 3 shopping malls (testing).
  • Ground Truth: High-density ground truth was generated using visual-inertial SLAM (via Google Tango) and manual geo-localization for long-duration mall sequences.
  • Synthetic Data: To improve the CNN's ability to generalize, 4,556 synthetic training samples were generated by "corrupting" clean trajectories with simulated sensor noise.

Results and Discussion

Fusion-DHL demonstrates significant improvements over existing methods in both accuracy and data density.

  • Superior Accuracy: Fusion-DHL achieved a Root Mean Square Error (RMSE) of roughly 5 meters, compared to the 12 meters typical of standard WiFi-based systems.
  • Massive Density: The system produces a 50Hz location history, which is 3,000 times denser than the 1/60Hz updates provided by standard energy-efficient APIs.
  • Robust Generalization: Even though the CNN was primarily trained on university building data, it generalized effectively to complex, unseen shopping mall environments.
  • Battery Efficiency: The system is energy-efficient because it primarily relies on the IMU—a low-power sensor—and only requires very infrequent WiFi scans to maintain global context.

Qualitative success is evident in visualizations where the CNN successfully "pushes" trajectories out of walls and keeps them within corridors and open retail spaces.

Conclusion and Future Work

While Fusion-DHL provides a high-fidelity reconstruction of indoor movement, it is currently designed for offline post-processing. It cannot yet be used for real-time, turn-by-turn indoor navigation. Additionally, the system's performance is sensitive to the initial alignment; if the initial match between IMU data and WiFi points is extremely poor, the CNN may struggle to find the correct walkable path. Future work will focus on adapting this multi-modal fusion approach for real-time indoor GPS applications.

Summary updated

Abstract

The paper proposes a multi-modal sensor fusion algorithm that fuses WiFi, IMU, and floorplan information to infer an accurate and dense location history in indoor environments. The algorithm uses 1) an inertial navigation algorithm to estimate a relative motion trajectory from IMU sensor data; 2) a WiFi-based localization API in industry to obtain positional constraints and geo-localize the trajectory; and 3) a convolutional neural network to refine the location history to be consistent with the floorplan. We have developed a data acquisition app to build a new dataset with WiFi, IMU, and floorplan data with ground-truth positions at 4 university buildings and 3 shopping malls. Our qualitative and quantitative evaluations demonstrate that the proposed system is able to produce twice as accurate and a few orders of magnitude denser location history than the current standard, while requiring minimal additional energy consumption. We will publicly share our code, data and models.
BibTeX
@inproceedings{herath2021fusion,
  author = {Sachini Herath and Saghar Irandoust and Bowen Chen and Yiming Qian and Pyojin Kim and Yasutaka Furukawa},
  title = {{Fusion-DHL: WiFi, IMU, and Floorplan Fusion for Dense History of Locations in Indoor Environments}},
  booktitle = {IEEE International Conference on Robotics and Automation},
  year = {2021},
  url = {https://openreview.net/forum?id=MYgfvZ2eNqG},
}
Figure